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The purpose of this research was to study problem solving 
processes used by elementary school boys. Boys between the 
ages of 7 and 12 years were administered several problems for 
three successive years. The total number of subjects examined 
was 800 and 35 problems were prepared for this study. 

Each problem administered was characterized considering 
its logical structure and its manner of presentation (language) . 
The results refer to problems built using three structures and 
three languages. The languages correspond to a pictorial pre- 
sentation (P) , ordinary verbal language (VA) , and abstract 
symbols (VB) . Since these two . comi'onerits are operationally 
independent, each structure was presented in each one of the 
three languages. For retesting at successive years, parallel 
sets of problems were used. 

The evaluation of a subject's performance was made con- 
sidering his tactics in solving a problem. A tactic is de- 
fined by the secpienceof questions that the subject asks in 
order to solve the. problem. The experimenter provides the 
answers that correspond to the questions the subject asks. 

A general theoretical framework was developed and the re- 
lationship between tactics and processes was formulated. The 
subject's tactics are scored considering the structure of the 
’ problems. Specific consideration is given to order, redundancy, 
amount of information obtained, etc. The "pulling out" method 
of scoring used in this research considers’ these variables. 

Notice that the scoring method used is independent of sample 
evaluations. The results obtained were treated statistically 
by a multivariate approach, specifically profile analysis. 

Subjects were classified in terms of age',’ I.Q. levels 
and grade point averages. On tlie basis of these classifications, 
several specific contrasts were examined. These refer to dif- 
ferentiation between structures, use of different languages, 
and effects of successive administrations on problem solving 
performance. Those contrasts were studied for several possible 
combinations of problems , for instance various structures and 
' ■ different languages; same structure and same language. 

The results indicate that there is a differentiation 
between structures that increases with age, but that at early 
ages this is better accomplished by problems in the P language. 

At around 9 to 10 years of age, language VA and VB become better 
discriminators. Several recommendations concerning psychological 
and educational problems have been made. A general conclusion 
is that in order to appraise thinking ability in problem solving 
performances , it is risky to rely exclusively on the final ans- 
of a problem as it is usually done, and that the differen— . 
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'tiation between logical structures find languages is basic. 

A subject may not be able to solve a problem, not because 
he does not understand its structure, but because he is not 
proficient in the use of the language in which it is pre- 
sented. Thus in each individual case, thresholds for both 
logical structures and languages should be established. 

It was further observed that in some cases discrimi 
nations between structures and between languages were re- 
flected in I.Q. levels, or in G.P.A. levels. Specific 
examples are given in the text. 



Introduction 



General Consiaerations 

The research to be reported in this volume concerns the exper- 
imental evaluation of problem solving tactics in children between 
seven and twelve years of age. The general design is longtitudinal 
since it studies the same subjects in three successive years. The 
results will be discussed with r' ference to age levels , independent 
evaluation of intelligence, and s. lool grades. 

The technique employed here to evaluate problem solving tactics 
departs from some of the approaches commonly used in this area of 
research. The major stress is in the experimentally controlled 
analysis of the process that -subjects follow when solving a problem 
rather than in the exclusive evaluation of the final answer . Con- 
sequently the operational procedures used depart in several respects 
from those used in other investigations of problem solving and 
cognitive processes. These differences can be more readily under- 
stood by making explicit the assumptions that underlie our approach 
and the properties "built in" the instruments employed. 

In many ways this research falls within the more inclusive 
chapter of cognitive studies. It may be properly qualified as 
epistemological in nature and development in design. Beyond 
studying independent groups of subjects at different age levels, 
we shall also experiment with the same subjects during a three 
year period. The findings that we shall report should complement 
those i-hat we have obtained in many years of research conducted 
with independent samples of subjects at different ages and 
educational levels. The references to this work can be found 
at the end of this chapter. Indeed, some of the major theoret- 
ical considerations that substantiate this investigation resulted 
from this previous experimental work. Therefore this research is 
an independent testing of experimentally derived hypotheses. 

Because of the type of experimentation that we have used, it 
is difficult to compare our results with those obtained by other 
investigators. Unless there is a reasonable communality in the 
assumptions made and in the operations performed, independent 
pieces of research may not alv^ays be successfully compared. As 
it is often the case, this results in endless argumentation after 
which no one gains necessarily in wisdom. In selecting the 
pertinent bi.bliography, we have limited the discussion to some 
of the recent work that is similar in assumptions, operations, 
and definitions to this study. If it is not always possible to 
conclude f"*om similar results of independent investigations that 
a point has been proved or a hypothesis verified, then neither 
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is it possible to conclude that different findings do represent 
divergent viewpoints. When in a restricted and a limited rield 
of research sufficient theoretical and operational sophistica ion 
has been reached, some of these issues may- be experimentally 
clarified by designing appropriate crucial experiments. Un- 
fortunately, crucial experiments do not abound in science an 
they are not always easy to plan. 

In order to obtain a better understanding of the issues 
involved in this research, we shall proceed by first cnaracter- 
izing some of the concepts used and second by discussing some 
of the theoretical and methodological issues involved in 
cognitive research. This will be followed by a statement of 
our assumptions, a description of the technique used, and the 
outline of the scoring procedures employed. Finally some 
theoretical and practical considerations will be presented with 
regard to the function played by logical structure and language 

in thinking processes. 

C haracteriza tionj3f__spme_te^ 

A cognitive process in a problem solving situation is defined 
as the sequence of psychological events and operations that occur 
when a subject solves a problem. This process begins with the 
presentation of a stimulus (problem or test item) . The final 
Llution (response) is an indication that the process has ended. 

R cognitive process is purposeful in the sense that it is ^ireote 
towards attaining an end or goal or aim, more specifically the 
solution of the problem. This moving towards a goal confers a 
dynamic character to the process. In the realisation of such 
directed activity the subject may aim at successive sub-goals, 
so that during the completion of the task (obtaining the so u ion 

detours may occur. 

The vicarious character of some cognitive processes is 
identified bv the substit'ition of certain expected elements 
and relations by others that are functionally equivalent with 
to gocil dtt9.imn6nt • 

Coanitive stylos may be described in terms of the goal , sub- 
goals, detours, elements, and relations. No doubt environmental 
Ld genetic variables play a very basic role in determining tte 
type of processes that subjects may preferentially follow. But 
thLe non-oognitive variables will not be explored in the present 

research. 

For the experimental observation of processes it is necessary 
to provide the subject with a situation such that he will 
to pursue the attainment of the goal by bringing into play those 
eluents that he considers necessary to solve a given problem. The 
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laws of c 'mposition that nold in a gfiven problGin, that is, its 
structural propertios, should be known prior to its administration. 
This allows a better understanding of the hunches of the hypotheses 
made and of how they are verified or rejected. It is obvious that 
in a concrete experimental situation it may be next to impossible 
to satisfy all these requirements. The technique used in this re- 
search satisfied some of them so that more important features of 
the subject's process could be observed. 

Theoretica l ^d_ matoqdological considerations involv ed in some 
cognitive research 

Regardless of their philosophical leanings, psychologists have 
explicitly or implicitly stressed the dynamic character of the 
cognitive process. Some of these are Binet's experimental study 
of intelligence (1902) , Spearman's formulation of his principles 
of cognition (1927) , Thurstone's description of his primary mental 
ability factors (1938), Duncker's (1945), and Wertheimer's (1945), 
studies of problem solving, as well as Koehler's (1927) experiments, 
and those of Heibreder and Maier. Though the conclusions may differ, 
a commoi. denominator that seems to run through all these researches 
refers to tlie dynamic character of cognitive performances and their 
purposefulness . A similar motive seems to run through more recent 
contributions as exemplified in Piaget's work, in Bartlett's ex- 
perimentation, in Vigotski's approach, or in Bruner's strategies, 
to cite but a few. V7e shall look at some theoretical and method- 
ological issues involved in the study of cognitive processes by 
analyzing them in terms of three major sub-groups: the experimental 

introspectionists , the testing approach and the observation of prob- 
lem solving behavior as preferred by the Wertheimer group. 

Besides the purely philosophical discussions , the experimental 
introspectionists have attempted to explore and characterize the 
ordered sequence of events implied in cognitive performance. In 
spite of the discredit that has unfortunately surrounded their 
efforts, it is fair to say that the deliberate effort made by these 
investigators to understand "what was going on in a subject’s mind", 
provided some systematic knowledge. As a matter of fact some of 
these issues are still actively discussed. For example Binet's 
examination of the problem solving performance of his two daughters 
is still rich in heuristic value. But the difficulty of repeating 
their observations and the strong interaction between subject and 
experimenter has diminished their general acceptance. 

VJith the advent and popularity of mental tests, these instru- 
ments became preferred tools for the investigation of thinking and 
problem solving and a new host of difficulties emerged. It is im- 
portant to consider: (a) characterization and selection of test 
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items or problems,: (b) evaluation of responses, and (c) inter- 
pretation of test results. 

A frequent approach to the characterization and selection 
of test items considers the proportion of subjects | responses that 
are right or wrong for a certain item. Seldom a discrimination is 
made according to the degree of correctness. On the basis of these 
results, items are either included or excluded in the final test 
form. This selection depends strongly on the sample used, so that 
for a different sample, different results may be expected. Subse- 
quent test scores are dependent on these specific sam.ple norms. At 
a later stage, the association between scores and independent criteria 
is determined to define the “thing that the test measures" . It is 
not exceptional to discover after a period of time that a particular 
test is not associated with the criterion that it was meant to apprais 

originally. 

For certain types of test items, it might be risky to use the 
procedures sketched above. For instance, let us take an item in 
which tile subject has to give an answer to the problem: 2 is to 8, 

as 3 is to There are many possible correct answers: for in- 

stance 9, (since 2 + 6 is equal to 8, and 3 + 6 equal to 9) or 12 
(since 2 x 4 is equal to 8 and 3 x 4 is equal to 12) , or 27 (since 
23 is equal to S, and 33 is equal to 27), or any even numbers (since 
8 is even) or any odd number (since 2 and 8 are even but 3 is odd) - 

or any number greater than 3 (since 8 is greater than 2) , etc. 

Obviously to score as correct the answer that is given with 
a certain predetermined frequency is not a criterion of correctness 
but only an indication of how m.uch subjects of a given sample agree 
in the response they give. In a different sample, another answer 
may be more or less popular, but this doesn’t make it either right 
or wrong. Niether does it make it right — or wrong — if m a large 
number of samples all the subjects would agree absolutely. Indeed, 
in the history of our civilization many a chapter deals with a 
reinterpretation or demonstration that some accepted correct answers 
or opinions were, after all, wrong. And in shewing this some few 
people paid dearly. 

Considering the score obtained by a subject, it is clear that 
the same performance may obtain different scores depending on the 
weights assigned to the items, since they in turn depend on the 
normative sample used. This affects the appraisal of a subject s 
ability and implies that the instrument is not invariant. Clearly 
the association that exists between scores and independent criteria 

also change as a result of this lack of invariance. 

This type of evaluation is of interest if we want to know how 
a subject stands in relation to a defined sample. But we may also 



be interested in knowing how to score the performance of a given 
subject regardless of the sample of subjects to which he belongs. 

This is more in lino with measurement as applied in most of the 
physical sciences and in every day life situations where let us 
say, the meaning of X centimeters is the same regardless of whether 
the object measured is wood or steel or of any other material. In 
this case the ruler (instrument used for measurement) appraises 
specifically the property of the object in which we are interested, 
and in this sense it is a valid measurement. But it also implies 
that the properties of the ruler (test items, problems) are well 
known prior to its application. 

The preceding paragraphs do not apply indiscriminately to all 
types of tests and are not meant to criticize the high level of 
ingenuity shov/n by many test theorists. In some cases the procedures 
that we have discussed are amply justified mainly in the case in which 
the test builder knows exactly what he wants to evaluate and how to do 
it. This refers essentially to instruments devised to appraise ac- 
quisition of certain specified types of knowledge, or achievement, etc. 
In these cases there is an attribute that we want to appraise, and 
since this attribute is (within limits) clearly defined, the construc- 
tion of a specific test (valid) does not present an insurmountable dif- 
ficulty. For instance if we are interested in appraising how much a 
subject knows about geography, no matter to which sample the subject 
belongs, saying that Latin America is north of the United States is 
wrongl But thi:’gs are otherwise when the subject has to ^^ngender a 
response that cannot be evaluated exclusively in terms of authority, 
or previous knov/ledge, or a reordering of previously known things. 

For instance in the previous example: 2 is to 8 as 3 is to it 

is one thing to use this as a way of knowing whether the subject can 
multiply and divide successfully and another to see what kind of a 
process he uses to reach the answer. In the first case items directly 
involved in multiplication and division will probably be more satis- 
factory. But also 13 may be acceptable since 3 is odd and 2 and 8 
are even. Or again 13 may be a good choice since (2x2) plus 4=8, 
and (3 x3) + 4 = 13. Even if all subjects would answer 13, a per- 
fectly objective answer, still \ie would not know hov? the subject 
reached it. Assuning that 13 is the only correct answer, we still do 
not knov7 which process V7as followed by each individual to reach such 
a response. That is, the end product does not necessarily specify 
the inferences and the processes that preceded it. In otliex words, 
to know what has occurred between the presentation of tx stimulus 
and the response may not always be safely inferred from the response. 

Most of the studies of cognitive processes in which tests of 
reasoning, intelligence, general ability, problem solving, etc., 
are used, ignore this vicariousness of the process and rely almost 
exclusively on inferences made from the observed responses. Since, 
the same response can be reached using different processes, then 
the inferences made are not as objective as more direct data on 
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processes. And this state of affairs is little improved by refining 
the scoring methods or the statistical analysis or by multiplying 
the number of tests administered or by comparing the average per- 
formances of different samples, and so forth. 

From the point of viev; of the psychology of individual dif- 
ferences, it is probable that by concentrating on the final answer 
we may be reducing our chances of discriiainating between subjects. 

If it is at all true that the same answer can be reached in dif- 
ferent ways, then the same final point can be reached by traveling 
along different paths. This convergence would reduce our possibility 
of differentiating between cognitive styles. In certain areas of 
knowledge it may be more important to know how a subject reaches a 
certain response than to know whether this response is or is not 
right. 

Hov/ever, it cannet be said that the study of final ansv/ers is 
irrelevant. The cxtrem.e opposite situation would correspond to the 
case in v;hich everybody would think straight but nobody would give 
a correct answer. That is, solving a problem necessitates both the 
process and the solution, and a technique for the study of problem 
solving should ideally allow for the characterization of the process 
and the identification of the response. If the technique used 
sacrifices one of these tvjo components, the results obtained are 
likely to be incomplete. In many practical situations, reaching 
a correct answer may be more important than following the best 
possible process. 

With regard to the traditional type of problem solving situ- 
ations , as analyzed by Duncker , Wertheimer , etc . , the stress has 
been on the side of the process. These studies are in a way the 
counterpart of those conducted by investigators that use predomi- 
nently the test approach and lack objectivity. The final response 
is not the main objective. On the contrary what subjects do and 
when they do it becom.es the central issue. This is made experi- 
mentally observable by asking the subject to verbalize his thoughts, 
or by looking at the way in which the elements of the problem are 
ordered and classified by him and so forth. In some cases the 
elements that the subject has to consider in solving the problem 
are rather complex, for instance connecting strings, folding paper 
or transferring liquid from one receptacle to another, or restoring 
equilibrium, etc. Other times the problems are presented using 
drawii\gs or ordinary language. In all cases, the problem solving 
situation terminates when the subject provides a final solution 
or elects not to proceed. 

Looking at these exper.iments , it is not always possible to 
define exactly the elements and logical system of relations built 
in the problems. Do they, for example, deal with binary systems, 
the absorption law, or antisymmetry, etc.? In some cases this 



could be done and would provide a guiding principle in evaluating 
subjects' performances. On the other hand a more restrictive 
definition of the formal properties may be less fruitful than a 
free approach for general exploratory purposes. But if the struc- 
ture of the problem is known and enough structures are explored ^ 
this shortcoming may be partially eliminated. 

More disturbing is the fact that an interaction may exist 
between the manner of presentation and the structure cf the 
problem. That is, we will not know whether not solving a problem 
is due to lack of understanding of the logical structure involved 
or to the subject's inability to operate with the concrete elements 
in. which the problem has been presented. Concluding that the subject 
in question cannot handle the formal relations implied in the prob- 
lem is an unwarranted assertii;n. But if the subject is given at 
least two problems with the same formal properties and different 
presentations (context) and if he is capable cf solving one of 
them, then it may be concluded that the subject understands the 
structure involved in the problem. 

It is indeed remarkable in fact that many every day problems 
and even routine performances may involve a complex network of 
relations and the subjects do this without major difficulty. But 
as soon as the same relational system is presented in abstract 
terms, few of them are able to solve it. The contrary situation 
also exists . The reasons why this occurs will not concern us in 
this monograph . 

Another point to be considered with regard to problem solving 
experiments refers to the interaction that may exist between the 
subject and the experimenter. This could be controlled by rigor- 
ously defining what is expected of the experimenter in general and 
in unexpected situations. Sometimes conclusions are reached 
concerning cogniti.ve performances in which these precautions are 
not strictly follovjed. It is then' difficult to separate what is 
due to the subject and what is due to the tester. Though from a 
clinical point cf view this may be acceptable, it is not so ex- 
perimentally. During tiie actual testing, the subject's per- 
formance should not be either hampered or facilitated by extra- 
neous remarks. Shifting the attention of the subject with help- 
ful hints or proddings may be recommendable in diagnostic or 
clinical situations or in some type of learning experiments, 
but in our context identity of conditions should hold for all 
cases examined. 

The results of these types of experiments are sometimes ex- 
pressed in an anecdotal manner. Eveu though recording procedures 
give evidence of precision, it is still extremely difficult or 
even impossible to elaborate and generalize about the process 
involved. 
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In this context, Duncker's original studj is exceptional 
considering the complexity of the situation involved. His efforts 
to differentiate what is observed from what is inferred at each 
step of the subject's or the experimenter's performance are well 
noted in the text. Unfortunately this is not always the case and 
the possibility of reproducing some modern studies in cognitive 
processes in a regorous experimental condition is, strictly speaking, 
very small. 

Description of the technique used to evaluate cognitive processe s 

in a problem solvincr situation 

■ . ■ . — _ 



The essential feature of the technique consists of presenting 
a problem that the subject has to solve by asking a series of (jues- 
tions. This represents almost a reversal of the usual testing 
situation, in the sense that here the subject searches for ques- 
tions that he thinks are appropriate to reach a solution. The ex- 
perimenter -"provides” a specific and fixed answer to each question 
asked. The subject is theoretically free to ask as many questions 
as he wants. Ratlier than being a passive receptor of stimuli, he 
is an active searcher of information, v^at he asks and when become 
features aiding an understcinding of his process. Ifhenever the sub- 
ject considers that he does not want to ask further questions, the 
testing situation is terminated. 

The sequence of questions asked by the subject defines his 
tactic. Observable variables that characterize the tactic are 
the number, type, and order of questions aslced. Further, since 
the solution offered is also available, it can be treated as a 
variable per se or in terms of its relation to the preceding 
sequence of questions asked. 

The technique was originally used in 1954 to evaluate medical 
diagnostic skills, and has since given rise to other forms. The 
plasticity of the technique makes it readily applicable to a large 
variety cf contexts fvor instance law cases, mathematical problems, 
everyday problem:-:, problems in chemistry or biology, psychiatric 
diagnosis, Rorsciiach interpretations, etc. In over fifteen years 
of research the Loyola Psychometric Laboratory and others have 
developed hundreds of instruments using this approach. 

In order to be consistent with our formulation of the problem, 
the subject should be able to ask any questions he wants vrhenever 
he wants. This poses some rather difficult problems for the ex- 
perimenter in the sense that it is impossible to predetermine the 
whole universe of questions that may be asked and to generate a 
set of answers corresponding to such questions. Similarly if the 
experimenter is left free to answer the subject's questions there 
is always the possibility of a subject-experimenter interaction. 



In order to avoid this, and to insure that a given question will 
always have the seime cinswer we use the following procedure: 

Each subject is presented a deck of 3 x 5 cards. On the first 
of these cards the problem is stated. Each of the remaining cards 
presents a question that the subject may ask. “Asking” a question 
means picking up a card. The pertinent answer is on the reverse 
side. Since each card is identified by a number, the experimenter 
or the subject can keep a record of the sequence of questions asked. 
Whether the problem is presented verbally or in writing, in some 
cases the possible questions correspond to drawings one drawing 
per card. For instance, assume that the problem consists of iden- 
tifying the shape and color of a box, knowing that there are round 
and square boxes, yellow and blue. One of the cards (questions) 
may contain a dravring of round blue and round yellow boxes. The 
subject at a certain stage in the process may want to know if the 
box is round, in which case pointing to the indicated card he will 
receive the corresponding answer. If the experimenter says yes, 
this means that the box is round (but still the color is unkno\'m) ; 
if the experimenter says no, then the subject should infer that the 
box has to be square. There are other possibilities for the mechanics 
of the asking and answering. For instance in Part III of the Test of 
the National Board of Medical Examiners (Hubbard, 1964) , the subject 
obtains the answe:: by erasing a rectangle next to the question so 
that the response v^ill appear. 



The problems used in this research can be seen in Appendix A. 
In all cases the instructions indicate that the solution of the 
problem may bo obtained by asking a certain series of questions. 
The subject is instructed to read the problem, examine all the 
questions and then to select only those he thinks are necessary 
for the solution. 

An objection to the procedure just outlined is that subjects 
do not generate their own questions but choose them among those 
presented to them. In earlier studies (Rimoldi, Haley, Fogliatto, 
1962a) , we examined the limitations introduced by this modus 
operandi. In developing the test of medical diagnostic skills, 
every subject was asked to list separately those questions they 
would like to ask, but that were not presented with the problem. 
Judging from the results obtained, the set of questions given 
covered the area satisfactorily. 



In another approach the subject was presented geometrical 
drawings in which he had to identify certain areas (Rimoldi, Devane, 
1961a; Rimoldi, Fogliatto, Haley, Reyes, Erdmann, Zacharia, 1962b; 
Rimoldi, Fogliatto, Erdmann, Donnelly, 1964). This he could do by 
asking self-generated questions. This presentation will not be 
used in this study. Clearly the type of problem that can be thus 
presented is liraited. 



Assumptions made i n the use of the technique 



These assumptions have been discussed in previous studies 
(Rimoldi, 1967; Rimoldi, 1969) and are summarized below. 

It is assumed that the tactic is an image of the process. 

To spell out tlie transf6rmation that holds between the domain 
of processes and the codomain of tactics is a very difficult 
task. If the correspondence is of the one-one type, then the 
»■ tactic is the exact counterpart of the process, so that the 

observation of the tactic gives exactly the same information 
that would be given by the observation of tlis process. In most 
situations this is not probably the case. 

A more reasonable assumption is that the ocrrespondence is 
of the many-one type., so that each process corresponds to one 
tactic though several processes may converge into the Scime tactic. 

In terms of our obser'/ations this would correspond to the situation 
in which the same observcibie behavior may result from different 
processes. In all likelihood this type of correspondence prevails 
in most situations. 

But if the correspondence is of the one-many type, then each 
process may originate e^ntirely different tactics, leading to dif- 
ferent solutions, and confusion v;ould prevail. From tliese assump- 
tions it follows that the experimental technique' that we use to 
evaluate processes requires a one-one or a many-one type of corres- 
pondence between processes as the domain and tactics as the co- 
domain. 

As previously stated, subjects are "witliin limits" free to 
ask questions. The major difficulty of this assiimption lies in 
the parenthetical expression "within limits" . The previous lesearch 
cited lends support to the validity of this assumption. It is im- 
portant to note that once a given question has been asked, the 
likelihood of asking or not asking one of the remaining questions 
will change. This will greatly depend on each subject, on the 
I questions that he has asked previously, on how he perceives the 

goal, etc. 



Another assumption is that tactics do reflect in some m.anner 
tliG information that the subject searches for, how he evaluates 
this information, how thoroughly he explores sub-goals and hunches, 
etc. This is a rather universal assumption in psychological re- 
search in the sense that observable behavior corresponds to covert 
behavior in some specified manner. 
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The same general considerations that were previously made to 
explain t^e correspondence between processes t^d ta.ctics may bo 
used to express the correspondence between specific events in 
the process ajid specific observations in the tactic. Notice that 
in the .case of tlie ma.ny**one relationship each event in the process 
would co.rrespond to one and only one event in the tactic, though 
several events in the process may converge into one observable 
event in the tactic. 

Lastly it seems much more fruitful to study the tactic rather 
than tlie final ansv;er. The tactic by definition is more complex and 
prior to the final answer. This should allow for greater differenti- 
ation between performances. 

The problem solving technique as used in this study 

' The specifications pertaining to the problems used in this 
study resulted from a variety of theoretical considerations and 
experimental results obtained in several separate studies. The 
formulation of the problems, the scoring procedures used, and the 
areas explored have gone through several changes. Meanwhile the 
1-iypothesas tested and the theoretical consieerations have come into 
sharper focus. This has allov/ed us to develop a plan for research 
in v?hich the contribution played by the formal properties of the 
problem and their moiiner of presentation can be experimentally 
isolated. 

Before describing the main features of the technique used in 
this study, a brief historical sketch of its development will be 
outlined. Throughout this developmental history, the following 
points will be considered the character irsation of (a) cognitive 
processes in a problem solving situation by the questions that 
a subject asks (tactic) , (b) the development, of a scoring pro- 
cedure to characterize ■*"he performance of a subject independent 
of a specific sample, (c) the more precise identification of what 
is "built in" the problems, that is, a more accurate control of 
tlie instruments used, (d) how these instruments differentiate 
between individuals, and (e) how tlie logical structure of a. 
problem and the m.anner of expression used (language) affect a 
subject's tactic — - and by influence, his processes -— so that 
cognitive sty.les may be described. 

Originally problems were built according to specific content 
areas like medical diagnostic skills (Rimoldi, Haley, Fogliatto, 
1962) , process of Rorschach evaluation (Tabor, 1959) , psycho- 
therapy processes (Gunn, 1961) , the Problem Solving and Information 
Apparatus (PSl) (John, Rimoldi, 1955), Training in Problem Solving 
(Rimoldi, Devane, 1961 ), mathematical problems (Rimoldi, 1967), 

Gtc« In essence we were experimenting with (a) the contents of 
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each prohl.em, (b) which manner of presentation was more appropriate 
for each. specific content area, for instance in the Test of Medical 
Diagnostic Skills careful consideration was given to whether the 
information should be given by writing, actual x-ray films, or 
photographs, etc« , (c) evaluation of tactics. 

In the early years we emalyzed independently the number of 
questions asked, the type of questions, and their order. The number 
of questions asked is a simple score to obtain though its meaning is 
rather complex. It was demonstrated thf?.t tliis score differentiated 
clearly between levels of training (Rimoldi, Haley, Fogliatto, 1962'; 
Rimoldi, Devane, 196] Rimoldi Fogliattc, Hpley, Reyes, Erdmann, 
Zacharia, 1962‘. ; Rimoldi, Fogliatto, Erdmann, Donnelly, 1964). It 
is still kept as an integral part of our scoring system. 

The utility index of each question was defined as the proportion 
of subjects that chose a given question in a given sample, the assump- 
tion being that if a question was perceived by the members of a group 
as more "useful’' than others, then it would be selected more often. 

It was possible to show that different questions had different 
utility indexes and that the same question may have different utility 
indexes depending on the educational level or other characteristics 
of the subjects in the Scimple being tested (Rimoldi, Haley, Fogliattc, 
1962; ; Rimoldi, Dc-var.e, 1961 ). Significant differences were related 
to expected Iv^vels of medical training, levels of education, etc. 

For a given sample, questions can be rank ordered in terms of 
the decreasing value of their utility indexes. Accumulating these 
rank ordesred utility indexes and plotting them against successive 
steps in the sequence, a maximum curve is obtained. This corresponds 
to the ideal perforraance that would represent a tactic in which the 
association is maximal between utility index and order in the se- 
quence. The performance curve corresponding to a specific tactic 
can be drawn on tht.'; Scime graph and compared with this maximum curve 
(Rimoldi, Devane , Haley,’ 1961 ). Plateaux would occur whenever- 
questions with a utility index of zero are chosen. The number and 
length of plateaus occurring in a given tactic was shevm to be a 
significant component ir differentiating problems and samples (Rimoldi, 
Fogliatto, Haley, Reyes, Erdmann, Zacharia, 1962. ; Rimoldi, Fogliatto, 
Erdmann, Donnelly, 1964; Haley, 1960). On the basis of those utility 
indexes, it was possible to differentiate the difficulty of problems 
(different problems, same sample of subjects) and level of samples 
(same problem, different samples) (Rimoldi, Haley, Fogliatto, 1962 
Rimoldi, Devane, 1961.; Rimoldi, Fogliatto, Haley, Reyes, Erdmann, 
Zacharia, 1962; ; Rimoldi, Fogliatto, Erdmann, Donnelly, 1964). But 
useful as they might be, utility indexes, strongly depend on sample 
norms and do not consider the order in which a question is asked. 



and thus they are weak indexes of the process. Still the remarkable 
association shown between this rough method of evaluation of per- 
formances and several independent criteria was a strong indication 
of the "validity” of the technique as a whole. 

In order to introduce the variable "order in the sequence" we 
explored the preferred location of a given question in a collection 
of tactics. It was found that some questions were preferrentially 
selected at the beginning of the process, others at middle or at 
the end of it, and still others almost anyv;here in the sequence. 

It was also found that this preferred location might vary with the 
sophistication of the subjects. The relationship between the util- 
ity index cf a question and its preferred location in the tactic 
was analyzed by coefficients as well as Kendall's W (coefficient 
of concordcince ; Rimoldi, Haley, Fogliatto, 1962- ■; Rimoldi, 1955). 

The method of pattern analysis was developed to investigate 
this order component (Rimoldi, Grib, 1960). But again, none of 
tb.esG procedures provided a "single" evaluation of the tactic. 
Nevertheless v/e V7sre becoming progressively aware that there v;ere 
other important variables related not only to the subject but more 
specifically to what "we put" in the problem. 

The next step towards a scoring system was to determine the 
frequency with which each question was selected in each possible 
order. This can be represented by tables in which columns represent 
questions and the row, order in the tactic. The entries in the cells 
correspond to frequency of selection. Since a question may be asked 
only once, performance can be represented using probabilities as de- 
fined in random sarApling v;ithout replacement (Feller, 1957). This 
means that if at a given step anyone of the possible questions is 
equally likely to occur, then at step k its probability of seclection 
will be 1 ■ v;here n is the total number of possible questions 
n-k + 1, 

presented with the problem and k is order U £ £ n) so at the first 
step we will obtain 3^ , and when k = n we obtain 1, that is the last 

n 

questions asked la completely determined. These values could then 
be used to test for independence between questions and order. This 
is an important consideration since we are assuming that tixe order of 
occurrence of a quo.'Jtion depends on what has proceeded it. In other 
words in the tabJ.es just described, covariance between questions and 
order plays an important role, and any statement based on scores 
obtained by using this information should cons.ider the association 
between questions and order. Nevertheless these va2.ues were ...sed 
as an approximation to score individual tactics, by accumulating 
the values corresponding to the questions asked in the order in 




18 - 



which they 3,re asked. This allowed the description of individual 
performance curves, including their length and the existence or 
non-existence of plateaux. We call this approach scoring using 
group norms. Clearly the procedure is heavily dependent on the 
sample used to define these values and violates some of the purposes 
that we initiated previously. 

On the other hand if all the members of a grc^up agree abso- 
lutely on the questions they ask and when they ask them, then the 
corresponding tabular representation covld .be given by a table in 
which in each row and in each coluirm there will be (n - 1) zero 
entries (for a problem with n questions) and only one filled in 
cell. That is, each question is asked in one and only one order 
by all the members of the group. This represents maximum agreement 
among subjects. The "chance agreement" would correspond to cells 
filled in using the 1 formulation. For descriptive purposes 

n-k+1 

an index of agreement can be defined using the transformation of 
information theor^^ (i^ttneave, 1959) so that maximum agreement 
would correspond to minimum uncertainty values. Then the perfor- 
mance of any sample of s’jbjects in a given problem can be readily 
compared in terms of its departure from this minimum uncertainty 
or vice versa in terms of its distance from the uncertainty value 
obtained under random sampling without replacement. These evalu- 
ations of agreement are strictly speaking, description of sample 
performances and do not say that a sequence that gives maximum 
agreement is necessarily the correct one. For instance assume 
questions a, b and c. Any of the sequences; a, b, c; a, c, b; 
b, c, a? b, a, c; c, a, b; and c, b, a, will give minimum un- 
certainty value if all the subjects follow this sequence exactly 
but only one of them may be, correct. This ap»>roach was exten- 
sively used in several studies (Rimoldi, Fogliatto, Haley, Reyes, 
Erdmann, Zacharia, 1962' ; Rimoldi, Fogliatto, Donnelly, 1964.; 
Rimoldi, Aghi, Burger, 1968" ; Vander Woude, 1969; Rimoldi, 1967) 
and in spite o.f its shortcomings can .be used to establish a 
relationship between agreement in group performance and its 
logic o;: approach. 

In studying how subjects solved certain mathematical problems, 
it was observed that rather complex logical relationships, like 
those in Newton ’o binomial or in the Pythagora's theorem could be 
dealt with if these structures were presented in a non-symbolic, 
everyday language. Of course, the children examined in that re- 
search did not know that in order to solve the given problem they 
were operating with the relational systems inherent to some complex 
logical formulation. As is often the case, the question is whether 
subjects can operate with a complex structure other than if they 
are aware that they are using logic or mathematics. 
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It became clear that if the structure of the problem was 
known then a scoring procedure based on this structure could 
be developed, that that this scoring procedure was independent 
of sample norms, though it could be used "a posteriori” to 
characterize samples. It was hypothesized that the same 

structure could be presented in different manners, so that a i 

given perfoinnance could be understood as a function of both 
the structure of the problem (intrinsic difficulty) and the 

manner of presentation or language used (extrinsic difficulty) i 

(Rimoldi, Haley, Fogliatto, Erdmann, 1963). Our most recent 
work has been based on the possibility of separating the logical 
structure of a problem from the language used. The problems ; 

used in this research were devised using these two concepts. 

Logical structur e and language in Jffie schem a pulling out method 

of scoring tactics i 

These two concepts have been extensively used in the study ; 

of cognitive processes but often it is difficult to say whether 
an experimenter is referring to one of them, to both or to neither 
of them. The logical structure of a problem can be defined by the 
system of relations that hold between certain elements in a defined 
set. By language is meant the meimer in which the logical structure ^ 

is presented provided a correspondence has been defined between the 
elements of the language and those of the logical structure. The ^ 

language may consist of words, abstract symbols, objects, etc. 

Since structure can be presented in several languages, these two 

concepts are in most cases operationally independent. ; 

Assume that the logical structure L (Figure 1) consists of : 

element a, 6/ Y with specified relations among themselves. 

The images of these elements can be obtained under different 0^ 
mappings. Each mapping represents a language so that 0^^ (L) = 

P., where is the problem built using language i. The corres- 
pondence between elements in the logical structure and elements 
in the 3..nguage can be specified. As indicated in Figure 1 th.e 
element a. in problem ’ 9 -^ corresponds to a many-one relationship 

so that a, is the image of both a and 0 in L when language 0-,^ is ' 

used. For another language (02) » ^2 corresponds to a and 3/ 

but the mapping of the logical structure is performed using language 

02* This generates problem p 2 * The element y h-s for image b^ 

under mappings 0, and b^ under mapping 02 / and similarly for c^ 

and c„. Under these conditions, problems and 9 ^ are isomorphic ' 

so ttiat a^-^a 2 / ^f^^2 problems used in | 

this study meet these specifications. In this sense a problem ^ 

can be defined as a mapping of a logical structure. 
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Figure 2a. Structure 31 




Figure 2b. Structure 33 
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Figure 2c. Structure 35 
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In a restricted sense, a thinking process can be understood 
as an attempt to make explicit and communicable to oneself or to 
others the formal properties of a problem (Rimoldi, 1967) . The 
tactic indicates how a given subject attempts to make explicit 
the logical structure of the problem. Depending on the subject, 
the logical structure and the language used, different tactics 
may be used. Clearly if the language used is not understood by 
the subject, then hie cannot solve the problem except by chance, 
and this can be observed in the tactic that he follows. Contrary- 
wise, if a logical structure is made explicit eind communicable in 
at least one language, then it can be said that the subject can 
operate with the formal structure involved in the problem. 

As the section on 2£ethod will indicate, several structures 
were used in this research. Each one. of them gave rise to several 
isomorphic problems presented in different languages. In that 
section several properties of the scoring methods are presented 
and in Figure 2 the structures are presented. The more specific 
approarb I'.sed for scoring these problems will be discussed in the 
sectiorx on K&thod. Scoring is a function of the number of questions, 
their type and their order and it is independent of any sampling 
evaluation. Indeed, tiie subject following the ideal tactic will 
obtain the maximum possible score. Ideal tactics are those in 
which the suJjject accumulates all the information needed to solve 
the problem, without redundancies, without order reversals, without 
irrelevant questions, and in the most parsim.onious manner. When- 
ever these requirements are violated, his score will be less than 
that corresponding to the ideal tactic. 

It is t-U 3 clear that the scoring procedure is independent 
of any sample evaluation. As a matter of fact the score that a 
subject obtains is exclusively the result of the structure of the 
instrument used and of his cibility to solve the problem.. Clv:urly 
subjects of different ability levels or with different personality 
characteristics, etc., will perform differently if the problems 
are sensitive to these variables. The method used tp score tactics 
that has been called schema pulling out method implies the following; 
1) we examine each tactic for redundancies, order reversals, etc., 
after having eliminated from such a tactic questions that are ir- 
relevant. By irrelevant we mean those that do not contribute 
information pertaining to the solution of the problem. After this 
has been done, a value is assigned to each one of the relevant 
questions, these values are combined and they are weighted by the 
total length of the tactic that the subject followed, including 
relevant and irrelevant questions as well. This is why the method 
has been called schem.a pulling out method. Irrelevant questions 
are first pulled out from the tactic and then are reintroduced so 
that the final results are weighted by the total number of questions 
asked. ' 
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In several studies in which this approach was followed, it 
was possible to obser\^e that a subject's tactic depends on both 
the structure of the problem and the language used. A factor 
analysis of problems (Rimoldi , 1970) indicated tliat factors 
corresponding to language and factors corresponding to logical 
structures could be clearly separated. Further, it has been 
shown previously (Rimoldi, Aghi, Burger, 1968) that if the 
structure of the problem is simple, then there is no interaction 
between language and age (from 7 to 13 years of age) but for more 
complex structures this is not the case. It was shown that at a 
certain age level certain new language take precedence over 
languages already well developed. Also (Vander Woude, 1969) it 
was shown that prclingually deaf children are able to solve 
relatively complex problems provided they are presented in a 
language that they ccin understeind. Finally the effects of 
language on performance in isomorphic probLems was shown for 
the ages 9 to 78. In all cases the problems presented in an 
abstract-symbolic language showed lower scores than problems 
presented in ordinary verbal Icinguage (Rimoldi, Vander Woude, 
1969) . 

General theoretical and methodological considerations 

The correspondence between processes a!iCl cs has been 

analyzed in previous pages. Among these we have defined the 
ideal tactic that follows closely the formal structure of the 
problem as given. If isomorphic problems (same structure, dif- 
ferent languages) are administered, then the contribution of 
the l.anguage used (extrinsic difficulty) and of the logical 
structure (intrinsic difficulty) can be separated. 

If the subject cannot understand the logical structure 
built in a problem, then no matter what language is used, his 
tactic will be widely separated from the ideal tactic. But if 
he does understcind the structure, then there will be a language 
in which he can express it. In the extrem ; case, if there is 
not a Icinguage and if he has enough ingenu ty, he might be able 
to invent a language in order to express the complexity of the 
structures involved. 

In terms of these concepts, it seems risky to conclude that 
a subject cannot understand a certain logical structure because 
he cannot solve a problem that implies such a structure. The 
reason may be simply that this subject does not understand the 
language used. 

With certain languages it is possible to express a greater 
variety of logical structures than with other languages (Rimoldi, 
1970; Rimoldi, 1967; Rimoldi, Aghi, Burger, 1968; Vander Woude, 
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1969; Rimoldi, 1969). This was observed in the factor analytic 
results reported above. Languages by means of which a great . 
number of logical structures can be expressed will be called 
onto languages. In this sense some languages approximate this 
onto property more than others. The usual spoken and written 
languages are of this type. Nevertheless, two languages may 
not be isomorphic and their u-egree of "ontoness" may vary. For 
instance it is well known how difficult it may be to translate 
very precisely concepts from one spoken language into another . 

It seems also intuitively clear that with signs, gestures, etc.. 
only certain structures can be expressed and not always with 
great precesion. In some cases it might be necessary to develop 
new symbols (for instance in mathematics and in sciences. in 
general) to express some structures more efficiently and without 
ambiguity. But these languages may not necessarily be appropriate 
to express other structures. 

This implies that besides their completeness (ontoness) 
languages may vary in precision. One-:one languages are by 
definition of complete precision, so that each element in ^e 
language represents one and only one element in the structure 
and vice versa. Incidentally, since one-one relationships imply 
the existence of an inverse, then with this particular type of 
language, whatever the subject does is the exact counterpart of 
whatever the subject thinks. This precision may be reached at 
the expense of reducing the number of logical structures that 
can be expressed. That is, certain structures can be better 
expressed using one specific language, but this specific 
language might be inappropriate to express other structures. 
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Methods 



Sample * , * . 

The sample* for this study consisted of ‘‘ViOS elementary .school 
boys between the ages of 7 and 12 ffom a large metropolitan center 
in the midwest of the U.S.A. ‘ They attended seven different Catholic 
schdols , -were predominantly of Polish descent, and lived in middle 
class neighborhoods. The selection of the subjects was made on 
the basis of responses received from the parents.' ' Letters ex- 
plaining the purpose of the study v;ere sent to the ‘parents of all 
the boy's of the above mentioned age. Parents' permission to test 
their boys was requested. Only those (boys) whose parents answere.d 
affirmatively were used for the study. 

As' indicated in Table 1, 705 subjects were tested during .tiie 
first year, 396 of these the second year and finally of these 396,’ 
352 were tested the third year. The dotted line in the table sep- 
arates 11 and 12 year olds in the first year of testing from the . 
earlier ages. The reason being that 11 and 12 year olds would be 
13 and 14 years old respectively in the second and third year of . 
testing and these ages were not contemplated in the design of the. 
study. However 11 and i2 year olds were tested in the first year- 
in order to establish a base line for further comparisons. The 
characteristic of the design partly explains the large difference 
in the' total number of subjects in the second' year of testing 
compared with that of the first year (from 705 to 396 of which 
256 were ll and 12 years old in the first year of testing) . 

Looking at Table 1 horizontally, it is seen that the original 
sample for each age is reduced approximately 10% in the successive 
years. This was due to reasons beyond our control such as subjects 
changing schools, moving out of the city and absenteeism at the 
time of testing. 

For every subject the following data was obtained from the 
school records: (1) Age from the nearest birthday to the time 

of the first year testing, (2) I.Q. All the schools administered 
Otis Intelligence Test as a routine procedure. The results of 
the latest testing were used to classify the students into high, 
medium and low using the criterion to be described later in the 
study. (3) Grades. Since all the schools offered to same subjects 
and had the same criterion for evaluation of student performance, 
the grade average for the first year of testing was obtained. The 



*We are indebted to Dr. Wozniak, Dean, School of Education, Loyola 
University and Sister Stephenette, C.S.S.F., Educational Consultant 
for the Schools directed by The Felician Sisters in Illinois, 
Lecturer, School of Education, Loyola University, for their indis- 
pensable and amicable cooperation in providing us the sample for 
this study. 
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First Year 



Third Year 



* .1 



Second Year 




Table 1 .. • * 

Description of the sample. In each line the top number refers 
•to age and the number in parenthesis (lower row) to-’nhffib'er- Of 
subjects. •• '• 
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academic courses on which the grade average is based were: Reading, 

English, Spelling, Handwriting, Arithmetic, History, Geography, 
Scierce, Christian Doctrine, Music and Art. (4) SAT scores (Standard 
Achievement Test). (5) Father's occupation: the following six 

categories were employed: , 

0 - Deceased father 

1 - Unemployed 

2 - Laborer, cook, etc. 

3 - Office worker, factory supervisor, etc. 

4 - Management, own business, etc. 

5 - Professional 

(6) Home conditions — that is, whether the child lived with both 
parents or comparable guardians: 

0 - Separated or divorced 

1 - Family intact 

These six items of information were coded together with the 
results of the testing for each student identified by school and 
nximber. All this information was put on IBM cards used for the 
statistical analyses. 

Problems 

The 27 problems basic to the analysis are presented in Table 
^ 2 ^ For s 3L, 33 cxnd 3S , are 3 languages : 

Pif,t-«j.e, Verbal A and Verbal B. For each of the 3 testing years, 
the problems are different except for those bracketed, that is, 

31VB]^ = 3 IVB 2 , 33VBj^ = 33 VB 2 / and 35VB^|^ = 35VB2. Problems of a 
given structxire are isomorphic except for the few exceptions 
mentioned below. A copy of each problem may be found in Appendix 
A. ■ ‘ 



Each problem consisted of 10 elements or questions, possibilities 
for which are illustrated as branches marked "+" designating available 
elements in a problem, along with the statement of the problem T^. 

The remainder of the 10 elements are irrelevant questions. 

■ Problems of the 31 series are represented in Figure 2. All 
nine of these- problems " are isomorphic . The elements marked "t" 
remain constant for each problem. Any combination of these marked 
questions constitutes a tactic, an attempt at solution. a^"^e 2 
represents the ideal tactic; ^ acceptable one, 

an impossible one for b^ is not available (no "+" ) . 

Of the 33 series, all but 33?2 and 33VA2 are isomorphic and 
fit the pattern in Figure 2. In these two cases at the risk of 



31 



let Year 



2nd Year 



3rd Year 



Structure 



33 



35 



31Pi 

31VAi 

plVBi] 


33Pi 

33VA1 

[33V^] 


35?! ' 

35VA1 

l35VBil 

■ V 


3IP2 


33P2 


35P2 


3IVA2 


33VA2 


35VA2 


plVB^] 


B3VB2] 


[35VB2] 


3IP3 


33F3 


35?3 


3IVA3 


33VA3 


35VA3 


3IVB3 


33VB3 


35VB3 



Tatle 2 

Problems used’ in- the study classified in terms of structure, 
language and year of administration. 
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not having strictly parallel problems across the years, an effort 
was made to assess the sensitivity of the procedure to slight changes 
in the presentation of the same structure. In 33?^ an additional 
element (f^ + hj)'*’ appears in the problem and also consequently 
there is one less irrelevant question. In 33VA2 two elements f2 and 
g2 were deleted and two irrelevant questions added. Despite these 
small var.iations, the ideal tactic is the same for all 33 series 
problems, that is aj^-<->b-|^-»‘i2 . 

In the 35 series, problems 35Pj^, 35VA^^, 35VA3, 35VBj^, 35VB2, 

and 35VB^ are isomorphic and represented in Figure 2. Notice that 

both d^"*" and (d + g )■*■ are given as well as (j^ + These 

z 2 2 2 ^ 

problems have four irrelevant questions. The three remaining 



problems of this series have the same structure though some of 



the given elements are different. ^^^2 ^^2 ^ ^2^^ instead 

of (d2 + g2) and k2^ additionally (only three irrelevant questions) . 

35VA2 uses , and j^'*’ instead of d^, i^r and (d2 + g2) • In 

35P-, e and (h^ + k are substituted for d~ and (d + g ). The 
3 2 2 2 ^22 

ideal tactic for all 35 series problems is ^ 



The following eight problems were also used but not included 
in the analyses, to date. Three seriation problems which involved 
simple orderings and three redundant picture problems were given 
the first and second years of testing. Redundant picture problems 
were parallel to the picture problems above except for the addition 
of a question that contained all the information presented in T , 
the statement of the problems. Problems 42 and 31A, not original 
problems for this study, were employed as a base line- to compare 
performance levels with previous researches. 



All these 35 problems are presented in full in Appendix A. 
Some additional problems similar to the ones above were also 
utilized as training devices but are not here described. 



Testing 

As indicated in the proposal, one of the main objectives of 
this research is to determine at what age and ability level children 
are able to solve problems of a specified logical structure with 
different modes of presentation (languages) . We attempted to 
determine the lowest age at which children are able to operate with 
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problems of known logical structure and mode of presentation. In 
a way this could be interpreted as trying to establish "thresholds 
for the two variables, structure and language. There were strong 
indications from our previous research (Rimoldi, Aghi, Burger, 

1968; Rimoldi, Vander Woude, 1968j Rimoldi, 1967) that at certain 
ages for problems with a given logical structure, the mode of pre- 
sentation that is most effective chcinges markedly. This is borne 
out by the fact that while at early ages problems presented with 
a specific language give satisfactory results; at a more advanced 
age the same problem is less satisfactory and sometimes inferior 
to another mode of presentation. 

It was hypothesized that the difficulty increased with the 
logical complexity of the problems, that is , 33 is more difficult 
than 31 and 35 is more difficult than 33. Also is was hypothesized 
that the picture problems were easier than the verbal ones and the 
verbal in turn easier than the symbolic ones. Further it was 
assumed that the seriation problems were the easiest. 



Previous research (Rimoldi, Aghi, Burger, 1968.) indicated 
that rhe length of the testing session should be about 1-1/2 hours 
long to gain and maintain the interest of the subjects. This time 
limit restricted the administration- * of all the problems to each 
subject. All these matters were considered in the testing design 
which is described below for each year. 



First year testin g 

a) Seven and eight year old group: _ 

Each subject was tested individually and was given a minimum 
of nine problems. The average session was 1 to 1-1/4 hours long. 

The subjects were first given the three seriation problems followed 
by a practice picture problem. After this the picture problems 3lP^, 
33P, and 35P, were administered in that order, followed by the 
redindant picture problems 3lPRj^, 33VR^ and 3SVR^. This was done 
for all subjects regardless of whether they were able to reach a 
successful solution. At this time if the subject performed well, 
the tester continued with the verbal problems, otherwise the testing 
session terminated. By "performing well" is meant that the subject 
on the whole followed either the ideal tactic or any of the good 
tactics (these terms will be explained in detail in the following 
section on scoring) . If it was decided to continue with the verbal 
problems, they were administered in the following order: 31A, 31A^, 

33A , 35A, , 31B , 33B,, 35B3^ and 42. If at any time after the first 
two^problLs, tie subject could not perform well in one of the prob- 
lems, the testing was terminated. 

b) Nine year old age group: 

Each subject was tested individually in sessions of about 1 to 
1-1/4 hours’ long. The subjects were first given a practice verbal 
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problem followed by 31A, 35h^, 33Bj^, 35Bj^ and 42 

in that order. If it was clear to the tester after administering 
two problems, that the svibject was failing to perform well, he 
was not given the remaining problems. Instead he was given the 
picture problems, , beginning with practice picture problems and 
followed by 31?^^, 33?^^, 35Pj^, 31PRj^, 33PRj^, 35PR^ in that order. 

If he still failed to perform well, the seriation problems were 
administered. 

c) Ten, eleven and twelve year old group; 

The order in which the problems were administered was the 
same as for age nine. The testing was performed, however, in 
groups of 10“20 instead of individually- If a subject's protocol 
indicated that he was not performing well , he was retested indi- 
vidually with the picture problems. The group sessions lasted 
about 1-1/2 hours. 

In summary, the thresholds for problem solving were estab- 
lished for testing all the subjects within a predetermined interval 
of hypothetical problem difficulty and then proceeding to a higher 
level if the subject was performing well or to a lower level if his 
performance was poor. 

Second year testi ng 

a) Seven* year old group: 

ZVgain each of these subjects was tested individually and was 
given a minimum of nine problems. The average session was 1 to 
1-1/4 hours long as in the first year of testing. The order in 
which the problems were administered was also the same; first 
the three seriation problems followed by three picture problems, 
3 IP 2 , 33 P 2 and 35 P 2 . Then the three redundant picture problems 
31PR^, 33 PR 2 and 35PR2 were given in that order. After this was 
done, the tester decided whether or not to continue with the 
verbal problems depending on the performance of the subjects. 

If it was decided to continue with the verbal problems, they 
were administered in the following order; first the practice 
verbal problem, then 331^2, 33^2, 35A2» 3 IB 2 / 33B2» 35B2 and 
42 in that order. Again the testing session terminated as 
soon as it was evident to the tester that the subject was not 
performing well. 

b) Eight year old group: 

Each subject was tested individually as during the first 
year of testing and the procedure and order of administration 
was identical to that during the first year. Thus the subjects 

♦For the sake of clarity, age groups are alway^J here identified 
by their first year age, for example here seven year olds are 
actual].y eight and also eight year olds would be ten in the 
third year of testing, etc. 
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were first given a practice verbal problem/ followed by problems 
31^2 / BBA^/ 35 x^ 2 r 3 IB 2 / 33 B 2 / 35 B 2 and 42 in that order. 

c) Nine and ten year old age group: 

The order in wh..ch the problems were administered was the same 
as in the previous year. This year the nine year olds like the ten 
year olds were tested in groups of 10“20 subjects. Again if the 
subject’s proto •'ol indicated a poor performance/ he was retested 
individually with the picture problems. 

Thus the problem solving thresholds were established for the 
second year also. 

Third year testing 

a) Seven year old group: 

Each subject was tested individually in sessions of about 1 
to 1-1/4 hours. The subjects were first given a practice verbal 
problem followed by problems llA^/ ^lA^? 35 .A 3 / 3 IB 2 / 33B / 35 B 3 
and 42 in that order. The procedure for determining whefeer to 
continue or to terminate the testing session was the same as 
during the first and second year of testing. 

b) Sight and nine year old group: 

The order of problems was the same as for the seven year old 
qroup. The testing was again performed in groups of 10—20 subjects. 
The group session lasted for about 1-1/2 to 2 hours as during e pre 

•vic' ;s two years of testing . 

Testers : 



The testing was accomplished by ten Loyola Psychometric 
Lciboratory male and female personnel who were at the level of 
advanced graduate students. Efforts were made to control indi- 
vidual-tester biases. All the testers were instructed and 
trained in live situations by an experienced member of the 
laboratory. In addition, the testers were generally assigned 
in a random fashion to individual subjects or groups of subjects 
of various ages over each of the three years of testing. The 
tester's function was to give preliminary instructions, explain 
a sample problem/ and present the test probl^s. If further 
instructions were necessary once testing commenced, deliberate 
efforts were made to rem.ain neutral, that iS/ neither to facilitate 
or hinder the subject's performance. 
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Scoring procedure 



The method used here is a refinement and a redefinition 
that resulted from the comparison of twenty different scoring 
methods in a Master's dissertation, by Mr.- T. Chlapecka. 

The purpose of this analysis was to design a scoring 
procedure that would be sensitive to the approach of the 
subject to the logical structure of the problem. The second 
end more difficult requisite was that this measure should have 
a significant degree of invariance to the superficial pecu- 
liarities of a problem, that is the number of questions, the 
ratio of relevant to irrelevant questions, etc. This means 
for example that a chance performance should have the same 
score in emy problem. Kore that score is set to zero, the 
score of the ideal tactic is set to 1.00, and the minimum score 
is -1.00. h score Si = .35 for example should reflect as 
accurately as possible the same degree of problem solving pro- 
ficiency regardless of the problem, and likewise for other 
values -liOO j< Si ^ 1.00. 

The approach followed considers how each tacric approx- 
imates the logical structure of the problem. By tactic we 
meain the sequence of questions that the subject asks. Approx- 
imating the logical structure involves asking the raore general 
questions first and thereafter questions of increasing specificity. 
The ideal tactic fulfills these requirements, that is: maximum 

correspondence between the generality of the question eind its 
position in the tactic with the minimum number of questions that 
exhaust the information necessary to solve tlie problem. In the 
scoring system used, these ideal tactics will obtain the maximum 
score. Scores are lower to the degree that they violate the 
above conditions, that is: reversals in order, irrelevancy and/or 

redundancy, lack of parsimony, failure to choose me 2 uiingful ques- 
tions, etc. 

The questions that the subject may ask can be classified into 
two major categories: relevant questions are those that provide 

information that is pertinent, irrelevant questions are those that 
provide no pertinent information. The relevant questions in turn 
can be further classified into subclasses defined by their degree 
of generality. With reference to Figure- 2 (a, b, c) in which the 
branches exemplify questions, each successive branch of the tree 
represents more specific questions. The degree of specificity is 
directly related to the magnitude of the subindexes. An order 
reversal will occur when a more specific question is asked prior 
to a more general question. A group of specific questions is 
equivalent to a more general question where the information 
embodied in them is the same as the information provided by the 
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more general cruestion. If a subject asks both a general question 
and some or all of the equivalent specific questions, those specific 
questions are considered redundant independently of their position 
relative to the general question. 

Assigning a score to an observed tactic involves several stages: 

1) Like in the ’'schema pulling cut method” (CRP 1449 and CRP 
2199) , all irrelevant and redundant questions are set aside from the 
observed tactic. This procedure reduces an observed tactic to one of 
the possible basic tactics. 

2) The Clements remaining in the basic tactic are then analyzed 

for order reversals. For instance, reference to Figure 2c indicates 
that problems built around structure 35 have two types of questions: 
a^, b^, and c^ of maximum generality and d 2 , ^2 ^2 

erality or greater specificity. 

In the scoring systera reversals of questions within each order 
of generality are not considered. Thus, for the case of Figure 2c, 
the sequences a^ , b^, c^, and c^, b^, a^ are identical and so are 

d 2 , ^2' ^2' ^2 ^2' ^2' ^2' ^2 forth. But ± 2 , 

implies reversals since the order of their occurrence in terms of 

tlie specifications set forth previously should be b^^, c^^, f 2 or 
c , b , f . The number of more general questions that appear in 
the tactic determines the number of positions in which the less 
general questions may occur. That is, questions b^^ and c^^ determine 
three possible positions for any specific question: either before, 

in between, or following them. If a specific question follows a 
general question, it is arbitrarily assigned a positional number of 
1. Other positions are assigned values related to the number of 
steps that they are reirioved from the "logical" order. So question 
f has the following values: 1 if Xn sequences c^, b^, f 2 or b^, 

c^, f 2 , 2/3 if in sequences b^^, f 2 / c^, or c^, f^, , and 1/3 if 
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in sequences f.b.c orf.c.b. 

2 11 2 1 1 

The general formula to determine these positional numbers is: 

a . = 1 “ D, where a . is positional number for a question p in 
PD J PD 

position j , j corresponds to the number of steps that the question 
p is removed form its "logical” order, and k is number of possible 
steps. This formulation can be extended to problems with any values 
of k, where 0 ;<j j^(k-l) . 

3 ) With tlie positional numbers as defined in 2 ) above, a matrix 
L is built in which the rows correspond to all the questions presented 
with a problem and the columns to the basic tactics as specified in 
1 ) 2dx)ve. In the cells of matrix L, the corresponding positional 
numbers are entered, the values for the irrelevant and redundant ques- 
tions being zero. An example of such matrix is given later. 



4 ) Each question in the problem is assigned a value in terms of 
the information it provides. Two conditions determine the information 
values (I) assigned to each question,. First, assigning a score of 
1.00 to the ideal tactic limits the information values of the x rele- 
vant questions. The more relevant questions in a problem, the smaller 
will be the information weight of each. The second condition relates 
the information weight of a general question to those cf its equiv- 
alent class. The information weight of a question at level t is 
defined as: 



It 



(r + 



r-1 



) I 



(t + 1) 



where r is the nijmbor of elements in the equivalent class (sub- 
branches) . For example in Figure 2 a has two sub-branches C2 and 



d2 (r= 2 ) , and Ii = ( 2 . 5 ) I2. 



A (1 X n) row vector W (W^, used to represent 

these information values for a given logical structure. A sample 
follows. Notice the information value of a given general question 
is greater than the sum of information values of its sub-branches 
by an amount directly related to the number of branches r. This 
process allows a weighting for parsimony or economy of a tactic. 
Consider in Figure 2 a the two tactics a^^, 02 and b2/ C2, 62* Both 

exhaust the information necessary to solve the problem, yet a,, - e_ 

a. z 

is more efficient and receives a higher score. A detailed rationale 
for this procedure may be found in the above mentioned thesis by T. 
Chlapecka. 

5 ) The score of basic tactic (x^) , where i ■ 1 m, can be 

represented as a linear function of the information weights and of 

the positional numbers so that, x. = w, a + w,, a + + w a .. 

1 I li ^ 2i n ni 

Similarly, for all the other basic tactics. 
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In the (n X m) matrix L, all the basic tactics corresponding 
to a given structure may be represented by the (m) columns, while 
the (n) rows correspond to all the questions in the problem. The 
(1 X n) row vector X, gives the score for all the basic tactics 
and is obtained by the nuatrix multiplication: 

X = WL 



In Figure 2a for problem 31P^ where questions 2,4,8 and 10 refer 
to C2» ^2 ^®spectively , we have: 



W = 






2 

236 



L = 



CO 

a 

o 

•H 

4J 

CO 

Q) 

3 

cx 



8 

9 

10 



1 

0 

0 

0 

0 

0 

1 



Questions p = (1, 2, n) 

3 4 5 6 7 8 9 

0 .714 0 0 0 .286 0 



10 



. 28 Tj 



Basic Tactics (i = 1,2, 



.m) 



4,10 10,4 2,8,10 4 8,10 2,8 2,10 2 8 10 



0 



1 

0 

0 

0 

0 

0 

.5 



0 



0 



0 

0 

0 

0 

0 

1 

0 

1 



1 

0 

0 

0 

0 

0 

0 



0 

0 

0 

0 

1 

0 

1 



0 

0 

0 

0 

1 

0 

0 



0 

0 

0 

0 

0 

0 

1 



0 

0 

0 

0 

0 



0 

0 

0 

0 

1 

0 

0 



0 

0 

0 

0 

0 

0 

1 



and X 



Basic Tactics 

4,10 10,4 2,8,10 4 8,10 2,8 2,10 2 8 

857 .758 .714 .572 .572 .572 .286 .286 



= [T.o . 



and likewise for problems built around other structures. 
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6) The final step analyzes the tactics for the use of 
redundcuit and/or irrelevant questions. The total amount of 
irrelevancy is set equal to -1.00 to be divided among the 
irrelevant questions of the problem. If a problem has three 
irrelevant questions ^ each one of them has a value Ir = -0.33. 

A relevant question is redundant when it appears in a 
tactic with a more general question covering the same information. 
Instead of receiving a positive information value, all the redundant 
possibilities share a total weight of -0.25. If there are two pos- 
sibilities as in Figure 2a for example, either or d 2 with a^, 
then C 2 or d 2 with would receive a weight of Rd = -0.125 instead 
of a positive information weight as would occur if the question 
were not in the tactic. 

The score for the observed tactic i is defined then as = 

+ Irj^ + Rd^, where -1.00 ^ Ir^^ ^ 0, and -0.25 ^ 

is the score of the basic tactic. 

Correlations between this new scoring system and the pre- 
viously used schema pulling out has been found to be, in several 
cases, of an order of magnitude of .90. Still the new method here 
described eliminates uncertainties inherent to previous procedures 
and has interesting theoretical and practical connotations. 

General manner of evaluation 

Multivariate analysis and more specifically Profile Analysis 
will be used throughout the study. In all cases the design can 
be presented as indicated in Table 3. 
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Table 3 

General Methodological Design 



Subjects 


1 


• • • • • 




..4 .1 . ‘ - -O 1 

• 

• 

• 

Sample 1 k 

• 

• 

• 
















1 

• 

• 

• 

Sample g 

• 

• 

• 

ng 


1 


t 

^kgb 












1 

• 

• 

• 

Sample r k 

- \ A ■ - • 

• 

nr 


• 


• 


* < *.! 

k K J 



where x,, means score of subject k in sample g and problem b,k- 

1....2...^g, g = l..,.2...r, and b = 1....2...t. 

The first hypothesis to be tested corresponds to the parallelism 
of group profiles and can be stated as follows: 
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^11 - *^12 

• 

• 




gi g2 

• 

• 






4 1 “ y o 

rl r2 

• 

• 


• 

^l(t-l) " ^It 




• 

^g(t-l) 


4 : 
gt 


1 


• 

^r(t-l)*' ^rt 



y , stands for the general designation for the population value 
corresponding to problem b in sample g. 

The second hypothesis to be tested corresponds to equality 
of sample levels. 

The third hypothesis refers to equality of problem levels. 

The computational procedure used follows the method pre- 
sented by Morrison (1967). i\nderson (1958) refers to a similar 
procedure . 

If parallelism is not acceptable; then the second and third 
hypothesGS will be tested respectively by t independentduaidattitd 
analyses of variance so that each variable b will be studied at 
the r sample levels i The hypothesis of equality of test means 
will, he analyzed using T^ test within each sample and for repeated 
measurements. Scheffe's (1953) simultaneous confidence intervals 
will be used to determine which one of the comparisons between 
samples or which one of the comparisons between tests are signif- 
icantly different. 

In the present study samples refer to either 1) age groups, 
or 2) I.Q. levels, or 3) grades, or 4) grade point averages. Prob 
lems will be studied considering two major varicibles: structure 

and language as indicated in j?ablG -'i below. 



Structure 





Same (isomorphic) 


Different 


Same 


• I Across’- years of 
testing for each 
s-;:ructure and 
each language 


II Across structure 
for each language 
and each year of 
testing 


Different 


III Across language 

for each structure 
and each year of 
testing. 


IV 



Table 4 
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Tho comparisons corresponding to cell I: same structure, same 

language are of the typei 



31P^- 


31 P 2 - 


3 IP 3 


31VA^ 


- 31 VA 2 


- 31VA^ 


31VB^ 


- 31 VB 2 


- 31VB^ 
3 


33P - 

1 


33P - 

2 


33P 
. 3 


33VAj^ 


- 33 VA 2 


- 33 VA 3 


33VBj^ 


- 33 VB 2 


- 33 VB 3 


35P3^ - 


35P2 - 


35 P 3 


35VAj^ 


- 35 VA 2 


- 35 VA 3 


35VBj^ 


- 35 VB 2 


- 35 VB 3 



These comparisons will indicate changes in scores as a function 
of first, second and third year of testing in isomorphic problems. 
These will be analyzed with reference to different samples. For 
each comparison paralellism, sample level and mean scores will be 
analyzed. The number of cases involved in each comparison will 
correspond to the number of subjects who were administered all 
these tests detailed in each of the comparisons tested above. 

The P problems deserve special consideration because in the third 
year they were given only to a limited group of subjects who were 
not c±>le to perform problems in the VA and VB languages. 

The comparisons implied in cell II : different structures , 

same languages, will be all performed for tests administered in 
the same testing session. 

These comparisons are: 

31P^ - 33Pj^ - 35Pj^ 

31VA^ - 33VA^ - 35VA^ 

31VB-^ - 33VBj^ - 35VBj^ 

3 IP 2 - 33p2 - 35 P 2 
3IVA2 - 33VA2 - 35VA2 

3IVB2 - 33VB2 - 35VB2 
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31P3 - 33P3 - 35P3 
31VA3 - 33VA3 - 35VA3 

31 VB 3 - 33 VB 3 - 35 VB 3 

Notice that in each comparison the language is constant but the 
structure varies. As in the previous case these comparisons will 
be performed for different samples. These comparisons will indicate 
the effects of structure on subject performances when language is 
kept constant. 

The comparisons in cell III: same structure, different 

languages should serve to indicate the effect of language on 
problems built arou.:-:. the same structure. According to the 
testing design, all three languages were not regularly ad- 
ministered to a subject each year. Consequently the size of 
the samples for those comparisons are reduced in many cases. 

Previous evidence (Rimoldi, Aghi, Burger, 1968; Rimoldi, 

Vender Woude, 1969) has shown the effects of language on per- 
formance in problems with the same structure. An inspection 
of the overall means corresponding to these types of comparisons 
can be done by inspecting Table 5. Here however, though the 
groups are comparable, the scores do not represent repeated 
measures . 

The comparisons in cell IV are not compaliible with the 
hypotheses of this study. 
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Results* 



This section will be organized in several parts. In Part I 
the analyses of the effects of chrcnolog.ical age on problem per- 
formance will be examined. In Part II the effects of different 
I.Q. levels will be discussed. In Part III each chronological 
age will be discusse*? separately and in Part IV problem per- 
formance will be examined in relation to grade point averages 
for each age group separately. 

In these analyses an attempt will be made to differentiate 
the effect of three major variables: language of the problems 
(P, VA and VB) , structure of the problems (31-33-35), and ad- 
ministration (lst-2nd-and 3rd) . The complete data on which 
these analyses are based are presented in Tables 5 to 20.* 

In Table 6 the overall mean and standard deviation for all 
the subjects of specified age groups are given for the three 
successive administrations of the picture problems. Notice that 
all problems with the same structure are isomorphic, except for 
those cases listed in the chapter on Method. In each block of 
Table 6 the number of subjects that were given each problem in 
each administration is also listed. 

Tables 7 and 8 follow the same pattern as Table 6, but 
refer respectively to problems in the VA and VB languages. 

The comments made with regard to Table 6 also hold here. 

Table 5 presents the means, standard deviations and number 
of subjects that were administered each problem in each testing 
session regardless of age. Tables 9 and 10 give respectively 
the means and standard deviations as well as the total number 
of subjects included in the multivariate analyses that were 
given problems in the P language. These are presented separately 
for subjects witli high (H) , medium (M) and low (L) I.Q. levels 
and for tiie total (T) . In some cases there were not enough sub- 
jects with.in a specified I.Q. level to perform the analyses. 

Still the results for the total sample are included in the table 
(for instance, 8 year old sample, 2nd administration) . 

Tables 11 and 12 and Tables 13 and 14 are similar to Tables 
9 and 10 but refer respectively to VA and VB problems. 

Tcibles 15, 16, 17, 18, 19, and 20 are similar to Tables 9 to 
14, the difference being that subjects have been classified in 
terms of their grade point average. 

In these tables all the basic statistics pertaining to this 
research are included. It should be noticed that the statistical 

♦Tables referred to here may be found starting on page 63. 
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tests performted in this study require the covariance matrix 
corresponding to the pertinent variables. The total number 
of s\±»jects used for this purpose will then depend on the 
number of subjects that at each age or ability level, or 
given grade point average (G.P.A.) were tested in "all" the 
problems of a specified type. This explains the discrepancies 
in the nuiober of subjects as they appear in the different 
tables here given. 

In Tables 21 to 34 the results obtained by testing the 
three hypotheses indicated in the section on Methods are 
given. Also the significance of the simultaneous confidence 
intervals for different types of contrasts are presented. In 
Table 21 the profiles cori'esponding to different ages are ana- 
lyzed. In Table 22 the profiles corresponding to the total 
sample subdivided into high, medium and low I.Q. levels cire 
examined. In Tables 23 to 28 the profiles for the high, medium 
and low I.Q. groups are studied for each age group separately. 

In Tables 29 to 34 the same analysis is made but the basis for 
the comparisons is G.P.A. levels. 

Each one of the Tables 21 to 34 has been arranged so that 
the upper half refers to cell II in Table 4. That is, the basic 
data is provided by the subject's scores as obtained in the same 
administration in problems expressed with the same language but 
different structures. The language of the problems used is given 
in the first coluiim and the administration session in the second 
column of these tables. 

The results of testing the three major hypotheses are indi- 
cated as fellows: 

1) In column 6, parallelism of the profiles. This indicates 
whether or not the average scores for different subsamples in prob- 
lems with different structures give’ parallel curves. 

2) In columns 3 to 5 inclusive the results indicate whether 
or not the profiles for each subsample are at significantly dif- 
ferent levels. 

3) In columns 7 to 9 inclusive the significance of the over- 
all uifference across structures is given together with the specific 
contrasts . 

Given that the profiles are parallel, the results of testing 
the second hypothesis indicate whether the classification into 
t.ie defined subsamples does madcc for significeuit differences in 
scores. The results of testing the third hypothesis indicate 
whether the different structures give results that are statisti- 
cally different. 
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The lower half of the tables refers to problems with the 
same language and the same structure in different administrations. 
Thus columns 3 to 5 inclusive give results that indicate whether 
or not the profiles corresponding to the subsamples used differ 
significantly. Columns 7 to 9 inclusive indicate whether or not 
administration produces a significant change in scores. 

Notice that the cells corresponding to columns 3 to 5 inclu- 
sive and 7 to 9 inclusive in Tables 21 to 34, two sets of values 
are given. Those in the upper half refer to the overall eval- 
uation of the contrast. _ Those in the lower half refer to specific 
comparisons as indicated at the top of the corresponding columns. 

With reference to the contrasts studied it should be remembered 
that the Scheffe's confidence intervals give very conservative mea- 
sures . 

In Appendix B a graphical representation of the content of 
Tables 21 and 22 is given. This supplementary material is * 
provided to help the reader visualize some of the results de- 
scribed below. For the tables presented here it is possible to 
construct similar graphs. 

Part I 



In this part the results obtained are discussed when studying 
the effects of age levels on problem performance. The samples used 
consist of tiiose subjects examined for the first time at 7, 8, 9, 
10, 11, and 12 years of age. These samples will be identified by 
a 7 year old Scurple, an 8 year old sample, a 9 year old sample, 
etc. These subjects will be one year older in the second admin- 
istration and two years older in the third adrainistration, so 
that the 9 year old sample in the third administration is of the 
same age as the 11 year old sample in the first administration. 

The information corresponding to this part is presented in Table 

21 . 



The main points to be discussed will be presented as a series 
of questions 1 to 4. These questions refer to how age levels re- 
late to performance considering language, structure and adirdnis- 
ration of the problems (1st, 2nd or 3rd administration) . In all 
cases we have made comparisons based on all the possible pairs 
of ages, as well as contrasts based on pooling together different 
age groups. 



er|c 
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For all the results here reported the profiles corresponding 
to specified age levels are statistically parallel,* 'I'his means 
that the relative differences in performemce are tho same for 
different age levels. 

Question 1 



How do scores in problems with different structures eind 
same language vary according to chronological age cind adminis- 
tration (1st, 2nd or 3rd year of testing)? These changes will 
be emalyzed for each language separately. 

With regard to problems 31P, 33P, and 35P in the first 
administration the overall significance for different age 
levels is at the .01 level, that is problems in language P 
do differentiate between age groups, regardless of the struc- 
ture used. More specifically these differences separate those 
subjects of 7, 8 and 9 years of age from those of 10, 11 and 
12 years at the .01 level. While there is no difference between 
any pairs of the ages 10, 11 and 12, or any pairs of the ages 7, 

8 and 9, 'iie comparisons (7-10) (8-10) and (9-10) are significant 
as indica. d in Table 21. Since the older group gives results 
that are s gnificantly greater than those of the younger group, 
it may be suggested that between 9 and 10 years cf age occurs 
a definite change in level of performance for the P problems. 

For the same problems among the second administration, 
only subjects in the 7 and 8 years of age samples were tested 
(at this admin j.str at ion the original 7 year old group is 8 years 
of age and the original 8 year old group is 9 years of age) . The 
reduction in Scimple size with regard to the first year of testing 
is due to the order followed in the test administration. As can 
be seen in Table 21 the two groups .-.re significantly different 
at the .05 level with the 7 year ola sample giving higher scores 
than the 8 year old sample. It should be considered that at 
this second administration the only 8 year olds that were given 
problems in the P language were those that performed badly in 
the VA and VB problems. Thus the fact that their performance 
is significantly lower than for the younger group is not sur- 
prising, and on the contrary it was expected. 

No comparisons are possible at the third year of testing 
since only the 7 year old sample (now of 9 years of age) was 
administered problems in P language. 



* Parallelism was tested at the .05 cind .01 level using 
D. L. Heck charts as presented by Morrison, 1967. 



with reference to the VA problems during the first adminis- 
tration, they were given to the 8 to 12 year old samples. The 
overall results indicate that the profiles corresponding to the 
different age groups have a significant difference in levels (.05). 
More specifically the significant differences at .10 level, corres- 
pond to the comparisons (10-12) and (10-11 and 12). That is, the 
difference in levels seems to occur between 10 and 11 years of 
age. Notice that the 8 year old group (represented by only five 
subjects) gives higher results than ciny of the other age samples. 
This may be explained considering the selection procedure used in 
the test administration (see Method) , that is these 8 year olds 
performed better tlian most of tlieir peers in the P problems so 
that they were given the VA problems. 

During the second administration of the VA problems, sub- 
jects from the 7, 8, 9 and 10 year old samples (now of 8, 9, 10 
and 11 years of age) were used. The differentiation between age 
levels is weak and separates the 10 year old sample from the 9 
and 8 year old Scimples. 

The third administration included 7, 8, 9 and 10 year old 
sample, now of 9, 10, 11 and 12 years of age respectively. The 
differentiation between age levels is at the .01 level, and occurs 
in the comparisons (7-9), (7-10), (8-.))^ (8-10) and (7, 8-9, 10). 
This indicates that a difference in perx'ormance occurs between 10 
and 11 years of age as was also the case in the previous adminis- 
tration. In summar^^ all the comparisons performed seem to indi- 
cate that at about 10 to 11 years of age there is definite im- 
provement in performance in the VA problems so that two relatively 
homogeneous subsamples could be defined below and above this age 
level . 

Only subjects of 9, 10, 11 and 12 years of age were tested 
during the first administration in VB problems. The overall test 
of significance shows a differentiation between ages at the .05 
level. It should be noticed that the 9 year old sample gave 
higher scores than the 10 year olds. The difference is signif- 
icant at .05. Again, because of the order in which the problems 
were a<ir.inistered, the 9 year olds for this analysis represent 
only approximately 50% of the 9 year olds who were capable of 
handling the verbal problems. The comparison (10-11, 12) is 
significant at the .05 level. 

During the second administration the VB problems differen- 
tiate at the .05 level between age groups. This overall dif- 
ference is due to the 8 year old sample and 10 year old sample 
(now of 9 and 11 years of age respectively) . 
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At the third administration the VB problems differentiate 
age levels at the .01 level. The Scunples studied are 7, 8, 9, 
and 10 v/hich now have 9, 10, 11 and 12 years of age respectively. 
The picture is similar to the one encountered with the VA problems. 
Inspection of Table 21 shows that tne comparisons (8-9) , (8-9) , 
and (7-10), are significant at the .05 or .01 level, while the 
comparison (7-9) is only significant at the .10 level. Pooling 
together the 7 and 8 year old sasiples and the 9 and 10 year old 
samples, a contrast that is significant at the .01 level is ob- 
tained. That is no difference between 7 and 8 year old samples 
or between 9 and 10 year old samples but a significeint change 
between the 8 and 9 year old samples, now respectively of 10 
cind 11 years of age. Again the suggestion is that between 1C 
and 11 years of age there does occur a significant change in 
per f ormcince . 

Thus , while problems in leinguage P suggest a change 
between 9 and 10 years of chronological age, for problems 
in languages VA and VB this change occurs somev;hat later, 
that is betv/een 10 and 11 years of age. It would be of in- 
terest to know how at later ages problems in P, VA eind VB 
languages behave. According to our previous experience, 

(Rimoldi, H. , Aghi, M. , Burger, G. , 1968) it may be hypothe- 
sized that the increase will be greater for the VA problems 
(and perhaps also for the VB problems) than for the P problems. 

It also looks, judging from our results, that for P prob- 
lems the differentiation is sharper at younger than at later 
ages, while the opposite seems to occur with the VA and VB 
languages. Therefore presentation of structures in VA and 
VB languages will differentiate better age levels in subjects 
above 10 or 11 years of age, while presentation of the same 
structures in P language will be more effective to differen- 
tiate earlier age levels. 

Question 2 



How do scores in the problems differentiate between struc" 
tures 31, 33 eind 35? 

The results given in the upper part of columns 31-33, 31- 
35 and 33-35 of Table 21 provide the pertinent answer. The 
consistency of the results is worth noticing. Regardless of 
the language used, the’ different structures give overall 
statistically significant results. In all cases but one 
(problems in P language, first administration) structure 33 
is significcintly different from structure 35. The difference 
between structures 31 cind 33 is significant only for problems 
in P language, first administration. With reference to struc- 
tures 31 and 35, in all cases the difference is significant at 



the .01 level, except for problems 31VA^ and 35VA^ (significant 
only at the .10 level) and problems 31P^ and 35P (significcint 
at the .05 level). These results were obtained oy pooling to- 
gether the age groups indicated in the columns at the right of 
Table 21. 

Inspection of the corresponding means (Table 6, 7 and 8) 
indicate that the scores in structure 35 are lower than those 
corresjjonding to structure 33. Similarly structure 31 gives 
always higher scores than structure 35 regardless of language 
used or administration (first, second or third) . Also there 
seems to be no difference between structures 31 and 33, except 
for problems in P language, first administration, where struc- 
ture 31 gives higher scores than structure 33. 

Considering that the scoring systen used does not depend 
on sample norms, but relates to the logical structure of the 
problem, these findings suggest that more thcui a purely ordinal 
evaluation of problem difficulty has been reached, provided the 
rationale used for developing the scoring system is accepted. 

In summary the comparisons over several age groups and 
language differentiate logical structures. In this study the 
indication is that structures 31 and 33 are "easier" than 35, 
with st3:ucture 31 and 33 of about the same difficulty level. 

The already noted exception refers to problems 31P^ and 33P^ 

This may be due to the fact that this is the first adminis- 
tration. Also, the sample used has a large number of 7 year 
old children v;ho might be able to understand significantly 
better structure 31 than structure 33 when both are presented 
in a perceptual context. At later ages or in successive ad- 
ministrations and with other languages, this difference tends 
to vcU'.ish as it may be indicated by our results. 

This suggests that for evaluating cognitive ability as 
well as for determining when a certain concept should be pre- 
sented within an education program, it might be worth considering: 
a) tlie logical structure of the concepts and b) most effective 
manner of presentation. Though logical structures and languages 
are within limits operationally independent, psychologically 
considered they probably interact, so that the language used 
to express a certain structure is not a matter of complete 
indifference. 

Question 3 

What changes in scores do occur in problems with the same 
language and the same structure for different age levels? 
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The corresponding findings are given in the lower part of 
Table 21. 

Since P problems were administered only to the 7 year 
olds for three successive years, no comparisons with other 
age groups are possible. Regarding the VA problems, graphs 
were made for problems of same structure, and same language 
the three successive administrations, separating them 
into age levels. The curves coffespond..ng to the different 
age scimples are parallel and their differences in levels are 
not statistically significant. In all cases the subjects 
of the lower age group give, higher scores than those of 
successively greater ages. This result is probably an artifact 
to the order in which the problems were administered. 

For the VB problems a similar result is obtained. Only 
two age groups were examined (0 year old and 9 year old samples) 
and the differences in level are significant at the .10 level. 

Question 4 



What changes in scores occur in problems with the same 
structure and the same language over years of testing (first, 
second and third administration)? 



here it is clear tnat in all cases administration does 
produce a significant change, whether the problems have struc- 
tures 31, 33 or 35 or whe^er they are presented in the VA or 
VB languages. T.he findings show that a significant difference 
occurs betr.-;een first and third and between second and third 
administration. The scores increase from first and second to 
third administration. 

Pa.rt II 



In tliis part of the report, we hall analyze the results 
obtained when subjects are classified into high, medium and 
low in terms of their I.Q. levels. This classification in- 
volved ail the subjects of a given age; that is, the top one 
third was tiie high group, the middle third was the medium 
group and the lower third the low group. The results referred 
to the total sample ignoring age levels so that the statements 
made do not differentiate between different ages. We shall 
compare the results for different structures and languages 
as well as for diffe.rent administrations. The results are 
given in Table 22. Though I.Q. levels vary slightly between 
samples, they are highly similar as can be seen by inspecting 
this table. 
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Notice that within any of the comparisons made, the corres- 
ponding profiles are always parallel, so that the curves for the 
high, medium and low groups show changes that are statistically 

similar. 

Question 1 

How do problems with different structures but expressed in 
the same language differentiate in terms of I.Q. levels and ad- 
ministration (first, second and tliird administration)? 

For the problems expressed in P language, the difference 
between I.Q. levels is significant at the .05 level during the 
first administration and only at the .10 level in the third ad- 
ministration. The results for the second administration are not 

statistically significant. 

In most of these comparisons the high group shows scores 
that are higher than the mediuia group which in turn shows 
Higher values than the low group. The only comparison that is 
significant for the P problems refers to ^e contrast between 
H and L group. That is, the P problems differentiate clearly 
between H and L groups during the first administration, but 
weakly during the third administration (.10 level). Since 
the sample of subjects that received the P problems is heavily 
loadr-d with subjects that below to the younger ages, this seems 
to indicate that the P problems differentiate mostly I.Q. levels 
in subjects at the lower ages. This is in general agreement 
with the previously discussed findings. 

VJith reference to the VA and VB problems the results are 
highly consistent. In the first place the overall level of 
significance is at the .01 level for all the administrations 
in all t.he problems. Secondly the high and low groups are 
also always significantly different at the .01 level. The 
difference between the high and medium group is always 
significant at that same level, except for problems of the 
VA series in the two administrations (.05 level) and for 
problems of the VB series in the third administration (.10 
level) The contrast between the medium and low groups is 
significant at the .01 or .05 levels except for the second 
administration . 

In summary, the P problems differentiate between extreme 
I.Q. levels mostly during the first administration (in which 
the 7 and 8 year olds represent 65% of the total sample) . 

In tlie second and third administrations, this difference is 
weak or non-existent. 
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On the other hemd the VA and VB problems do differentiate 
between any of the possible combinations of I»Q. levels in all 
administrations, except for the VA and VB problems in the second 
administration, which nevertheless approach significance. 

Notice that the highest level of significance for the 
three I.Q. comparisons occur in problems of the VA and VB 
types in tlie first administration. In this administration 
the age groups 10, 11 and 12 year olds represent approximately 
85% of the total sample (see Table 21) . It seems reasonable 
to infer that the VA and VB problems do differentiate most 
effectively between I.Q. levels at the high age groups while 
the P problems are most effective in differentiating I.Q. 
levels at the younger ages. 

Question 2 



How do scores in the problems differentiate between 
structures? 

As shown in Table 22 the results are highly consistent 
throughout different languages and administrations. The over- 
all significance betv/een structures is always at the .01 level, 
except for problems in the VA language during the first ad- 
ministration (.05 level). The general pattern is quite similar 
to the or.e discussed when studying age levels (Table 21) . This 
in itself is an interesting finding since in spite of the fact 
tliat the subiects involved are the same (whan studyiiig age levels 
and when studying I.Q. levels) the particular pattern of signif- 
icant differences did not have necessarily to agree. The fact 
that this has happened is to be noticed. It seems to indicate 
that regardless of tlie two classification systems used, struc- 
tures show the sam.e differentiating power.* 

Question 3 

What changes do occur in problems in the same language 
and with tlie same structure as a result of different I.Q. 
levels? 

This implies studying the performance in three problems 
with the same language and same structure through three 
successive administrations. In the first place neither problems 



* Note that the niutjber of subjects in Table 21 are very 
slightly lower than those in Table 22. When the claasi 
fication of the subjects was made with regard to age, 
those age groups with N < 2 were not included. 
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of the 31P or 33P series differentiate among I.Q« levels. Prob- 
lems of the 35P type differentiate only at the .10 level between 
the high and low groups. The pattern for the VA and VB problems 
are highly similar. IThether they correspond to structures 31, 

33 or 35, all of them give overall significant differences be- 
tween I.Q. levels (.01) and in all cases the comparison H-L is 
also significant at that level. Only problems of tiie 31VA type 
differentiate between medium and low I.Q. levels (.01). The 
contrast H-M is significant at the .01 level for all problems 
except for those of series 31VA and 33VB. 

These findings summarize the results of the cornparrisons 
based on the three administrations of problems with the same 
structure and same language and indicate that within idie same 
administration, perfontiance in the problems relate to I.Q. 
levels so that si±>jects with higher I.Q. 's will in general 
perform better than f^.ose with a low I.Q. level. It should 
be remen±)ered tliat when the same type of comparisons were 
performed with regard to age levels, the results were as- 
sent ial].y negative. 

It is worth noticing that with respect to Question 1, 
the significance between levels was due to the H-L, H-M, 
and L-M comparisons throughout different structures. In the 
case that v;e are new discussing the contrasts are analyzed 
through administrations and the significant M-L coraparison 
disappears (except for problems of the 31VA type). Again 
it seeiT's that for the purposes of differentiation among 
I.Q. levels it may be more sensitive to compare subjects 
performances in terms of several structures than to ad- 
minister several isomorphic problems in successic .. 

Question 4 

What changes in scores occur in problems with the same 
structure and language as a result of administration, when 
these changes are referred to I.Q. levels? 

For the VA and VB problems the results are almost exactly 
tlie same as those reported in Table 21. The P problems follow 
the same pattern that the VA and VB problems follow, except 
for those of the 31P series. Here the overall difference is 
at the .05 levrol and due primarily to the contrast between the 
sGCond 2ind third administration. 

In all cases performance in the third administration is 
higher than in the first or second administration. The second 
administration may or may not give higher scores than the first 
administration . 
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Part III 



In this part we shall analyze the results corresponding 
to each age group separately with regard to I.Q. levels. The 
influence of structures and years of testing will be discussed. 

By age group we mean the age of the subject the first 
time that he was tested. Thus the chronological age at a 
particular administration corresponds to the age group plus 
one or two years respectively for the .second and third years 
of testing. For each age group a table will be presented 
summarizing the' results obtained. 



7 year old sample (Table 23) 

The results are based on those subjects v/ho were ad- 
ministered the three problems used for comparison, that is 
either isomorphic problems through '•.he three years of testing 
or the three dif.ferent structures presented in the same lan- 
guage. Because of these conditions as well as because of the 
order in which the problems were administered the total number 
of subjects in each comparison is not constant. 



With reference to problems based on different structures 
but expressed in tlie same language, we find that in all cases 
(except for the probleits 31P , 33P and 35P^) subjects' per- 
formance differentiates wea/cly between I.Q. levels and that 
this is on.lv due to the contrast between the high and low I.Q. 
groups. The significance of this contrast is higher for the 
va problems in the third administration, that is when the 
average age of tlie subject tested wa', 9 years of age. 



Notice that for all the comparisons the profiles corres- 
ponding to the three I.Q. levels are parallel. This implies 
that the changes observed among the subjects belonging to a 
specified I.Q. level are not statistically significant from 
the changes corresponding to subjects at a different I.Q. 
level . 

With regard to structures the same pattern of significant 
values that is shown is Table 22 has been found. That is, the 
overall significance reaches the .01 level in all the problems 
administered to the 7 year old sample except for the P problems, 
third administration. More specifically, the significant dif- 
ferences refer to the comparisons between struc..ures 31 and 
33 and 33 and 35. The difference (31-33) is significant at 
the .01 level only in problems presented in P language, first 
administration. The lack of significance in P problems, third 



administration, indicates that at this age level (9 years of 
age) subjects that have been tested twice before, perform 
equally well on all the structures involved in the comparison. 

It is interesting to notice that this does not occur for prob~ 
lems of the VA and VB type during the same third administration. 
The possibility that this lack of differentiation with the P 
problems may result from a leveling off in performance (plus 
the influence of possible training effects) may not be dismissed. 
That is, the subjects at this age do understand the three struc- 
tures involved in the three problems and can operate equally well 
in any one of them if presented in P language. 

Comparing now problems with the same structure and the 
same language at different administrations, it is found that 
across administrations there are no significant differences 
due to I.Q- levels in P problems with structure 31 and 33. 
However, with structure 35 the results indicate that the 
subjects with high I.Q. level perform better (.10 level) 
than those with a low I.Q* level. 

Comparing now level of performance during the three 
successive administrations, we found, that for all the struc- 
tures the third administration gives results significantly 
better than the first or second testing sessions except for 
problems in the 31 series. That is, scores in structure 31 
expressed in P language change little with subsequent ad- 
ministrations . 

Notice that for the 7 year old sample the cells corres- 
ponding to first and second administration in VA and VB 
languages are empty. This is due to the fact that very few 
of the 7 year olds were administered these problems either 
the first or the secgnd year of testing. For the same reason 
the comparisons of problems in VA and VB languages with the 
same structure but different administrations could not be 
performed. 

In summary, the general co:uments made in the previous 
sections of this study seem to apply to the 7 year old sample. 

8 year old sample (Table 24) 

Only very few subjects of this Scunple were administered 
the P problems in the second and third administration, that is 
when their chronological ages were respectively 9 and 10 years 
of age. At the same time they received VA and VB problems 
during the second cind third administration. 
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Comparing pcirformances in problems 33P]^, eurid 35 Pq^, 

for the three different I.Q. levels, we observe that the over- 
all significant difference occurs at the .10 level and is due 
only to the comparison of the high and low I.Q. groups. For 
the VA and VB problems the comparison of I.Q. levels gives an 
overall significance at the .01 level in all cas^, as well as 
for the specific high- low contrast. 

The comparisons i *.dium-low are only signific^t at the 
.10 level and occur ix. ooth VA and VB problems during the 
second administration. The high-medium contrast is signif- 
icant at the .01 level in problems of series VA 3 , otherwise 
the contrast is vjeak. Tiiat is, the pattern is quite similar 
to the one reported for the previous analyses. 

For all the problems administered to the 8 year old sample 
the overall compsrison through structures is always significant. 
The differences betaveen sttuctures 31-35 are in all cases signif- 
icant at the .01 level, except for problems in language P, first 
administration, where the overall difference is significant at 
the .10 level and due to the contrast between stniatures 31 and 
33. Again as in previous cases, structure 33 is significantly 
different from structure 35 in the VA problems in the second 
and third administrations. 

As can be observed in the corresponding table for tlis 8 
year sample, there were no comparisons made in any problem 
during three successive years. This was a result of tlie 
design ured in the administration of the problems. Again 
the general pattern agrees with the previous observations. 



9 year old sample (Table 25) 

As can be seen in the corresponding table the P problems 
during the first administration do not differentiate in terms 
of I.Q. levels. The VA problems differentiate I.Q. levels in 
the first (. 10 ) level) and third (.05 level) administrations. 
The VB problems differentiate I.Q. levels only in the second 
administration (.05 level). Notice' that at this age level the 
P problems were only administered if the subjects performed 
poorly in tlie problems administered previously. This contrib- 
utes to the explanation of the lack of differention of I.Q. 
levels found with the P problems . For the VA problems during 
the first administration , ^ the I.Q. level differentiation is 
weak, and for the VB problems it is not significant. Never- 
theless, for the 8 .year old sample, second administration 
(9 years of age) the results were highly signif5.cant. It 
may be postulated that the VB problems are too difficult at 
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this age level while with the VA language the subjects may 
have some greater facility. Still this does not explain why 
the VA2 problems (that correspond to 10 years of chronological 
age) and the (that cor.resj^ond to 11 years of chronological 

age) do not differentiate between I.Q. levels. The inspection 
of the means involved in these contrasts does not help much, 
and we can not now offer ein explamation. 



With regard to structures, the same pattern that was pre- 
viously found becomes again quite apparent. The comparison is 
always significant for the contrasts 31-35 and 33-35. In the 
case of the P problems, it is only the comparison 31-33 that 
reaches a .05 level of significeince. 

For problems with the same structure and language it is 
found that those of the series 31VA do not separate between 
I.Q. levels, while those of series 33VA and 35VA do. The pro- 
file for problems 33VA was not parallel and thus three uni- 
variate cinalyses of variance were performed showing that it 
is only at the third administration (11 years of chronological 
age) that the structures are significantly different. This 
can be readily seen by inspecting the corresponding graphs. 

Whilw problems of the series 33VB and 35VB do not dif- 
ferentiate between I.Q. levels, problems of the ..series 31VB 
do. Again the profiles are not pa.rallel and the analyses of 
variance show- that during the first administration, I.Q. 
levels are significantly different at the .01 level, while 
at the third administration the level reached is only .10. 



With regard to years of testing, notice that the pattern 
previously found is repeated here. This is, scores increase 
from first to second to third year of testing. In the com- 
parisons in which non-parallel profiles are involved, that is 
VA33 ar.'u VB31, Hottellings’ T^ test shows that the changes in 
scores with administration are significant at the .01 level 
for the medium and high groups. For problems of the series 
VB31 the lew I.Q. group shows significant changes between 
administrations . 



With regard to this 9 year old sample the results 
share many features in common with tliose previously found. 
However, the picture is less sharp than for younger or older 
groups. The subjects here studied are those that were orig- 
inally of 9 years of age. At the second and third adminis- 
tration their ages will be respectively 10 and 11 years of 
age. As previously hinted, it is at this age level that 
some basic changes se€im to occur with regard to the effective 
use of the languages involved in this research and with the 



understanding of the structures. Therefore the findings here 
reported do show the effects that such a transition period is 
most likely to produce. 

10 year old sample (Table 26) 

Only a small sample of 10 year olds had to be administered 
problems 31P, 33P and 35P. Inspection of the corresponding 
graphs shows a high level of performance for the profiles corres- 
ponding to the three I»Q» levels which are not significantly dif- 
ferent. Nevertheless the VA 2 ind VB problems show in all cases 
(first? second? or third administration) differences in the pro- 
files corresponding to different I.Q. levels. These are signif- 
icant at the .01 level. This is mostly due to the contrast high- 
low I.O. The- second most predominant contrast refers to the 
coniparison H-M, especially noticeable in the second and third 
administration vdien the subjects were respectively of 11 and 
12 years of age. Notice that the comparison medium- low I.Q. is 
significant in both VA and VB problems during the first adminis- 
tration. The similarity of these patterns of differences with 
the overall pattern indicated in Table 22, is intriguing in the 
sense that vfhile in the majority of the cases the significance 
of the differences between I.Q. levels depends mostly or. xdie 
H and L I.Q. groups, it is only at the older age levels that 
the differences between high and medium I.Q. groups and medium 
and low I.Q. groups begin to emerge. This of course relates 
to the differentiating power of the structures and languages 
used and their relation to age. 

At this age level the P problems do not differentiate 
among s'*'-ructures , the implication being that for all -the 
problems uscvd the performcince is at a high level regardless 
of the str-acture used. Nevertheless for the VA and VB prob- 
lems the results indicate not significant differences with 
regard to structure during the first administration. During 
the second and third administration the overall test of 
significance shows that structures are differentiated .at 
the second and third administration. As always the pre- 
dominant contrasts refer to structures 31-35 and 33-35 re- 
peating the overall findings given in Teible 22. For this 
10 year old sampl.e second and third administration correspond 
respective.ly to 11 and 12 years of age. 

Comparing problems with the same structure and language 
in the three administrations, it is found that in all cases 
the comparisons are significant (.01) and that the specific 
contrast high- low I.Q. is also significant at the same level. 

For the VB problems the differentiation is also significant 
between the high and medium I.Q* group*, in 'all the structures. 
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For the VA problems, structures 33 and 35 behave similarly. 

With regard to administration notice that except for 
problems VB 31 all the others provide significant results 
indicating that there is alv/ays a significant difference in 
the scores obtained in the firsw and third administration 
as well as in the second and third ones. 

The picture given by the 10 year olds is definitely 
clearer than the one shown by the 9 year old group. It 
appears that at this level (which includes subjects of 10, 

11 and 12 years of age) differentiation of I.Q. levels, 
structures, adridnistration and languages becomes well 
delineated. 

11 year old sample (Tc^le 27) 

Problems VA and VB were administered only once according 
to the design (TaJ::>le 1) . Thus comparisons through differnt 
testing sessions will not be made. 

The results for problems of the VA type and VB type 
indicate that they differentiate successfully between I.Q. 
levels (.01) and that tlie H-L contrast is significcint at 
that sarr.e level. 

At this age level, the performance in the different 
structures gives significant results (.05 level) due mostly 
to the sigrdficant contrast bet^.^reen structures 31 and 35. 
Notice that the scores for these problems are high. 

12 ye a r old sa-iple (Table 28) 



The general comments made with regard to the 11 year 
old sazpl.e also hold here. The results repeat the same 
general picture already discussed. The VA problems dif- 
ferentiate between I.Q. levels (.01).' The significant 
contj.'asts er to the comparisons between high-medium, 
cind high-].ow I.Q. levels. The VB problems differentiate 
sjjtnilarly, including the comparison medium- low that is 
significant at tlie .10 level. 

With regard t.o structures, problems 33 and 35 are 
clearly differentiated in both VA and VB problems. The 
contrast 31-35 is significant (.05) only in problems in 
the VB language. 
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structures better than grade point average, at the second and 
third year of testing (9 and 10 years of age respectively) . 
Otherwise the patter; 5 are highly similar. 

9 year old sample (Table 31) 

For the 9 year old sample the contrasts between structures 
are the same in terms of G.P.A. and I.Q. levels. Similarly with 
regeord to contrasts involving administration in problems with the 
same language and same structure (isomorphic problems) . Notice 
that the lack of parallelism found for VA33 and VB31 disappears 
with using G.P.A. as a criterion for classification. Columns 3 
to 5 inclusive in Table 31 show that for P problems there is no 
difference in the samples based on I.Q. levels or in G.P.A. 
levels. For the VA i^roblems the classification in terms of 
G.P.A. scores differentiates better than the I.Q. level classifi- 
cation when we consider the contrasts that involve problems with 
the same language and different structures. And a similar result 
is observed for the VB language. 

Comparison of isomorphic problems through administration 
does not give a clear picture. At this age level, the pattern 
that had prevailed in previous years seems to be shifting in 
the sense that VA problems become better discriminators in 
profiles based on G.P.A. scores than in profiles based on I.Q. 
levels. It is appropriate to remember that the 9 year old 
sample includes subjects of 9, 10 and 11 years of age for the 
first, second and third adniinistration. It should also be 
remembered that it was at the 9 year old sample that the 
pattern of contrasts found for I.Q. levels was not clear. In 
summary it seems that this age sample corresponds to a transition 
per lod in which the increasing influence of VA IcU? Tuage seems to 
be the predominant factor. Apparently th.e P language has here 
less differentiating power. 

10 year old sampl e (Table 32) 

For the 3.0 year old sample, classification of siat'jects 
in terms of G.P.A. scores gives substantially the same results 
as classification in terms of I.Q. levels. The most noticeable 
difference is that in contrasts involving the same language and 
different structures as well as in contrasts involving isomorphic 
problems, there is a predominence of significant differences 
between high and mediura and medium and low levels when classifying 
subjects in terms of grade point averages. For the I.Q. classifi- 
cation tlie differences were mostly in terms of high-low and high- 
medium levels. This may suggest that grades in school tend to 
differentiate more sharply among the lower achievers while I.Q. 
will separate more clearly subjects in the high and medium levels. 




11 year old sample (Table 33) 

For the 11 year old sample, the picture given by the 
based on G.P.A. levels is almost exactly the same 
as the one given by the I.Q. profiles. 

12 year old sample (Table 34) 

As in the case or the 11 year old sample, no major dif~ 
ferences cire found between classifications based on I.Q. and 
G.P.A. levels. 
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Table 7. Verbal A Problem Means, Standard Deviations 
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Table 8. Verbal B Problem Means, Standard Deviations 
and Niimber for the Total Sample 
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Table 20. Verbal B Problem Standard Deviation for Multivariate Analyses 
Total Age Groups Divided into High, Medium, and Low G.P.A. Groups 
Age at Different Administrations .-, ^ ^ 
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Table 21. Multivariate Analysis Results: 
Comparisons of Different Age Groups 
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Table 22. Multivariate Analysis Results: 
Comparisons of High, Medium, emd Low I.Q. Groups Over the Total Sample 
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Table 23. Multivariate Analysi:; Resuirs: 

Comparisons of High, Medium, and Low I.Q. Groups for 7 Year Olds 
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Tcibie 24. Multivariate Analysis Results: 

Comparisons of High, Medium, and Lov/ I.Q« Groups for 8 Year Olds 
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Table 25. Multivariate Analysis Results: 
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Table 26. Multivariate Analysis Results: 

Comparisons of High, Medium, and Low I.Q. Groups for 10 Year Olds 
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Table 27. Huitivariate J^nalysis Results: 

Comparisons of High, Medium, and Low I.Q. Groups for 11 Year Olds 
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Table 28. Multivariate Analysis Results: 

Comparisons of High, Medium, and Low I.Q. Groups for 12 Year Olds 
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Table 29. Multivariate Analysis Results: 
Comparisons of High, Medium, and Low G.P.A. Groups for 7 Year Olds 
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Table 30. Multivariate Analysis Results: 
Comparisons of High, Medium, and Low G.P.A. Groups for 8 Year Olds 
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Table 31. Multivariate Analysis Results: 



Comparisons of High, Medium, and Low G.P.A. Groups for 9 Year Olds 




Table 32. Multivariate Analysis Results: 

Comparisons of High, Medium, and Low G.P.A. Groups for 10 Year Olds 
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Table 33. Multivariate Analysis Results: 

Comoarisons of High, Medium, cuid Low G.P.A. Groups for 11 Year Olds 
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Table 34. Multivariate Analysis Results: 
Comparisons of High, Medium, and Low G.P.A. Groups for 12 Year Olds 
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Conclusions cuid Recommendations 



In attempting to interpret and summarize the results of 
this study, it is necessary to highlight some of the unique 
features of this research. These refer to theoretical eind 
methodological considerations as well as to the experimental 
procedures used. 

With regard to theoretical considerations we attempted 
to separate the concepts of logical sttucture and language 
and subsequently to develop a procedi;.re that would permit 
tlieir experimental identification. This resulted in the 
preparation of instruments (problems) strictly defined, so 
that in terms of these two components, language and logical 
structure, a classification of problems could be made. 
Isomorphic problems were defined as resulting from mapping 
the same logical structures using different manners of ex- 
pression. 

If the languages used are different (P, VA, VB) a series 
of isomorphic problems could be of the type 31P, 31VA, 31VB, 
and similarly for other structures. This corresponds to 
those problems described in cell III of Table 4. If the 
structure, and the language are the same but the words or 
symbols used are different, the series would be represented 
by the sequence (for instance when using the VA language 
using different words) 31VA.., dlVA^, which 1, 2, 

3, refer to administration but also to the concrete manner 
of presentation of the problem.- These correspond to the 
problems described in cell I of Table 4. It could be said 
that problems of the 1st type are, as it were, translations 
of the Scune structure in different languages, while problems 
of the 2nd type are synonymous. 

It follows that the comparison of performance in prob- 
lems like those of cell III in Table 4 should give informati-'n 
as to how language affects the subject's tactics for a speci- 
fied structure. The comparison of problems like those corres- 
ponding to cell I in Table 4 will give indication as to how 

the use of a given language changes through successive years. 

/ 

If a subject solves at least one of all the isomorphic 
problems administered to him, then we are entitled to conclude 
that he can deal with the corresponding logical structure. By 
using isomorphic problems it i.s then possible to establish 
thresholds for the different structures. If a subject cem 
solve a problem with structure 31, then he is at or above the 
threshold corresponding to that structure. On the other hand 
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if he can solve problem 31P, but not problem 31VA, then we 
could say that though he is at or above threshold for struc- 
ture 31, he can not yet operate successfully with the VA 
language. Of course this can only refer to the special 
words used in presenting problem 31VA since a synonymous 
problem may be solved. In this research, synonymous prob- 
lems were used to study the effects of successive adminis- 
trations . 

If a subject can operate satisfactorily with a given 
language, then comparison of problems presented in the 
same language but built around different structures should 
give information concerning his understanding of the logical 
structures involved. Problems of this type correspond to 
those of cell II in Table 4, for instance 31P, 33P, 35P. 

The different comparisons implied in the previous 
description were performed in this research. Their cross 
examination provides a check on each specific finding so 
that we could test thresholds for logical structure through 
performances in isomorphic problems (translations and 
synonymous problems) as well as using one language and dif- 
ferent structures. Similarly, language thresholds can be 
checked by examining when a subject can solve a problem 
in a given language. For instance if a subject can solve 
problem 33VA, but not 33VB, then we will say that he is 
below threshold for Icinguage VB. 

The comparisons perfomed in this research are given 
in Tables 21 to 34. They were performed independently using 
several subclassifications of subjects, namely chronological 
age, I.Q., and G.P.A. The similarities or differences ob- 
tained should then be of interest to appraise the influence 
of these varicibles in problem solving performances. Further, 
changes in successive performances (first, second and third 
administration) were performed for problems with the same 
language and same structure. 

As explained in che Introduction ,to this study, we 
examine subject’s tactics, a tactic beii.g defined experi- 
mentally by the sequence of questions that a subject asks. 

It is assumed that tactics are images of the subject's 
processes (the correspondence being either of the one-one 
type or of the many-one type) . Usually in studying cognitive 
processes , experimenters rely on the final answers given to 
problems, and as our experience has shown, it is risky to 
infer processes from final answers. On the other hand a 
thinking aloud approach, or other similar experimental 
situations make it difficult to compare results, mostly 
when different researchers are involved. 
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Therefore the problems used in this study are a com- 
promise between a situation that leaves both subject and 
experimenter a considerable degree of freedom and a situation 
in which the interpretation of the responses may be widely 
divergent from what subjects really do. As stated previously 
our main interest is in the study of tactics / and these are 
made observable through the analyses of the questions that 
the subject asks. 

This' brings into the foreground methodological problems 
that do not weigh so heavily in other types of tests. One 
of them refers to the order in which questions are asked. In 
years past v;e scored problems in terms of indexes developed 
from the performances of selected samples. But as discussed 
in the Introduction, the same performance may obtaiii different 
evaluations according to the normative sample used. On the 
other hand it might be desirable to score subjects' tactics 
regard] ess of the san\ple to which he belongs. Thus we have 
attempted through the years to develop a scoring procedure 
such that it will depend exclusively on the properties of 
the problem used 3nd on the subject's ability, as expressed 
in his tactic. 

In the section on Method this approach is described in 
detail. Basically scores depend on how the subject handles 
the logical structure of the. problem. The ideal tactic is 
the one that follows the logical structure closely and most 
parsimoniously without reversals or redundancies. The result 
is that sco.res can be assigned to specific tactics based 
exclusively on how a subject approaches the logical structure 
of the problem. Thus isomorphic problems should obtain the 
same pulling out score regardless of the language used, pro- 
vided the tactic is the seme in all of them. Otherwise the 
observed pulling out score will vary, and these changes in 
score should reflect the "difficulty" of the problem as 
presented. The concept of "intrinsic" difficulty of a prob- 
lem relates to its logical structure. The concept of extrinsic 
difficulty depends on the language used. If two isomorphic 
problems give two different pulling out scores in the Scime 
subject, then we are entitled to say that die difference is 
due to the language used; Thus for a give structure the 
intrinsic difficulty is constant no matter which language 
is used. 

The pulling out scoring method used in this study is 
an improvement on the previous pulling out scoring methods 
that were used in the past. The procedure used here allows 
the comparison of scores for different structures along a 
continuum, that varies from -1.00 to 1.00. In all likelihood 
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the method that we use here goes beyond a simple rank ordering 
of values. If this is demonstrated then logical structures can 
be placed in terms of their difficulty within the interval 
indicated above. It is intuitively clear by analyzing the 
results of this research, that this condition has probably been 
fulfilled. 

It should also be noticed that the scoring method used 
here considers as a very important variable the relation of 
procedenc.e among questions, a temporal context this means 
that logical structures can be examined psychologically as 
temporarily ordered events. 

The fact that we are not using sampling procedures to 
evaluate tactics does not mean that statistical operations 
can not be performed to define samples cmd evaluate subjects. 

As a matter of fact the type of score that is used here is 
similar to the one used in those measurements for which a 
ruler has been defined. Applying such ruler to an object, 
results (scores) can be used to classify the observations 
into different subsets and the corresponding statistics can 
be pc^foianed. In the present research the properties of 
the ruler used are defined in terms of logical concepts. 
Basically these refer to the properties of ordered sets. 

The statistical analyses performed could have been 
more extensive and other methods could have been used. 

The vast amount of data accumulated required a procedure 
that would allow dealing with several varicibles simultaneously. 
Further since the same subjects were tested at least three 
times, covariances play a definite role in the interpretation 
of the results. Therefore early in our research it was decided 
to use a multivariate analysis appraoch, more specifically pro- 
file cinalysis for several independent groups. The independent 
groups refer to the classification of subjects in terms of 
chronological age, or I.Q., or G.P.A. averages. The covariance 
matrices refer to successive administration of the same prob- 
lems or of isomorphic problems or of problems varying in both 
language and structure as the case may be. An important facet 
of this study relates to the study of specified contrasts (age 
groups, I.Q. levels, G.P.A. levels, etc,.). The Scheffe tech- 
nique was used for determining these simultaneous confidence 
intervals . 

For completeness of information we have included summary 
statistics for all the problems used in this study. These 
appear in Table 5. Other analyses could have been performed 
to discuss our data. Nevertheless because of the purposes of 
the study and the straight forwardness of the multivariate 



- 96 - 



o 



formulation, the procedures used were thought to fit our aims 
best. 



The experimental procedure used with regard to the ad- 
ministration of the problems implies testing simultaneously 
for both structure and language thresholds. Those subjects 
who could not solve problems in P language were not ad- 
ministered VA or VB problems. Similarly subjects who at 
later ages could not solve VA problems were administered 
P problems. On the basis of our previous research an in- 
creasing difficulty level was postulated progressing from 
P to VA to VB for languages and from 31 to 33 to 35 for 
structures. The results of this study seem to indicate 
that our "guess" was essentially correct, at least within 
the age groups considered in this investigation. But by 
doing this, and by not using those subjects that did not 
meet the minimum requirements specified in the section on 
Method, the sample used was at or above threshold at least 
for the simple structure (31) and for the P language. Never- 
theless the differential thresholds between structures and 
languages could be tested. As shown in the section on 
Results, they were in both cases significant (Table 21). It 
seems quite clear that between 9 and 10 years of age there is 
a significant change in level of performance when using P 
language. A similar change is observed approximately a year 
later with VA and VB languages. These results seem to indicate 
that for the age groups used in this study, two relatively 
homogeneous subsamples can be defined with reference to 
language P; one including ages 7, 8 and 9, and another in- 
cluding ages 10, 11 and 12. With the VA and VB languages 
a similar phenomenon is observed approximately one year 
later. Further it was suggested that with P language dif- 
ferentiation of problems tends to become less clear at later 
ages, though with the VA and VB languages the opposite seems 
to be true. The total sample, including all ages has shown 
that structures are differentiated in all the languages used. 
Also that contrasts based on administration session, for prob- 
lems with same structure and language, show that the scores 
increase from first and second administration to third ad- 
ministration. Thus throughout the years there is a progressive 
improvement in successive performances, when VA and VB languages 
are used. Nevertheless the comparisons between age levels for 
problems in the same language and same structure through the 
three years of testing do not give significantly different 
profiles. That is, while comparison of age levels is made on 
the basis of problems with saiae language but different struc- 
tures (keeping administration constant) the age levels are 
clearly differentiated but not so when the problems have the 
same language and the same structure. 
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The educational and psychological implications of this 
obs-vation are of interest, in the sense that the presentation 
of a given concept may be more readily understood in one la»9"3ge 
than in another. That is given that a subject or ^ ® . 
reached the threshold that corresponds to a given logical struc 
ture, then it should be a question of presenting it in 
guage that is most appropriate, ft systematic analysis of thres 
holds of different logical structures should be conducted an 
recommendations concerning the most effective ma^er of pre- 
sentation should be an integral part of such study. But in ^ 
doing so the burden of the proof should be left on 
in the sense that the student's tactics, that is how they 
a problem, should tell us how each individual 

This is obviouslv different from appraising a student s cJJ y 
by noting how many problems he answered 

In this situation no tactic is involved, and possible detours, 
failures in reasoning, missing the crucial elements, simple 
adhering to a. pattern, etc., cannot be discovered' except 
in roundabout ways . 

Once these thresholds have been determined it should be a 
relatively simple matter to find out which language or manner 
of presentation increases the possibility of discriminating 
among structures. Our results seem to indicate that the P 
language is better fitted to fulfill this purpose in the early 
years , but becomes less discriminating with increasing age. 

With the Vft and VB languages discrimination gets better with 

advancing ago, 

our results also suggest that average evaluations may 
risk ignoring the outstanding and the backward 
in the case of those 7 and 8 year olds that perforaed better 

than their peers. In this respect _t -b v 

since our scoring system does not depend on sampling pe 
formanoes, it would be possible to place a ^b^^^ect in a 
specified category regardless of the age group to which he 
belongs. Hi.s score will depend exclusively on how he under- 
stands the logical structure of the proolom. 

With regard to profiles based on I.Q. levels (Table 22) 
comparing the performance of problems with same language but 
SfLent structures in the same administration, the results 
indicate that different I.Q. levels give 

ferent results. This is basically due to problems in Vft and 
VB languages and much less in P problems, structures are as 
1^1 dlffLentiated as they were in the case of age levels. 
However the comparison of I.Q- profiles using problems wito 
the same language and same structure through the three yeM_ 
of testing shows highly significant results, with ^^f- 

ferentiation between H-L and H-K contrasts. Also the third 
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administration gives higher results than the first or the 
second’- .-ones. Pooling together this information and com- 
paring it with the one obtained using an independent criterion 
for classification (age level) suggests some interesting pos- 
sibilities. One of these is that I.Q. levels may depend quite 
heavi2.y on the language used. 

That is, inability to operate with a given language may 
show in reduced test scores, not because the subjects are un- 
able, to understand the structure of the problem but because 
they are less proficient in the use of the manner of expression 
in idiich the problem is .presented. These observations verify 
previous findings listed in the Introductory section. The im- 
plication being that in studies concerning language deficiency, 
perceptual deficiency, eft.c., it is not enough to conclude that 
the cognitive thinking ability of a subject is low because of- 
his failure to solve one or several problems. The questions 
should rather be, 1) does he understand the logical structure 
of the problems, 2) can he use the language efficiently. A 
negative answer to the first question is quite final, but a 
negative einswer to -the second one does not s^ much about the 
first. This, of course, has implication in interCulturAl studies. 
As a matter of fact, fluency in a language is not an indication 
that the subject can understand complex or even simple logical 
structur.es . 

Comparisons of profiles in age or levels with prob- 

lens of the type 31VA^, 33VA^, 35VA^, where i is any fixed ad- 
ministration, were shown in general to be significantly dif- 
ferent. The same type of comparisons of profiles defined by 
isomorphic problems,, for instance 3 IVA 2 / ^^^^2 ^IVA^ do 
not show significantly different levels when the profiles are 
based on age, -though in terms of I.Q. levels they are signi- 
ficant. In all cases, whether using I.Q. or age, the com- 
parisons between structures is significant as well as the 
contrasts involving administration v structure 31 and 33 are 
overall significantly different from structure 35 and the 
third administration gives better scores than the first or 
the second) . Pooling together these results it seems rea- 
sonable to infer that in order to appraise a subject’s 
ability it may be better to scan his performance in several 
problems with different structures than to administer to 
him several isomorphic problems. The question -that beeches 
then focal is: should students be trained by practicing 

repeatedly with isomorphic problems, a few selected concepts, 
or should they be administered problems with different struc- 
tures in a lauiguage that they can mi^ipulate. Though it is 
true that educationally speaking, a certain minimum level of 
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proficiency is desirable, the danger may reside in the pos- 
sibility that by concentrating on this aspect the subjects 
with outstanding ability may be ignored. As it is well known 
this has considerable implications in education and in society 
in general. This seems to fall in line with some of the ques- 
tions that some psychologists and educators are raising today 
with regard to some testing procedures commonly used to evaluate 
cognitive ability and thinking processes. It is then suggested 
that a large variety of problems with different logi^-^.l com- 
plexity should be used to evaluate subjects' ability. But un- 
fortunately if the structure of these problems is made in terms 
of how a group of subjects perform, then the risk is that the 
most discriminating problems (test it^s) will either be elim- 
inated or would obtain a low weight in the final evaluation. 

The existence of a transitional period with regard to the 
use of language seems to occur between 9 and 11 years of age. 

The pattern of significant contrasts before and after these 
ages seems to indicate that there is a shift from" the use of P 
language towards the more efficient use of VA and VB languages 
at the. later years. This is shown whether subjects are classi- 
fied in terms of I.Q. levels or of G.P.A. scores. In terms of 
the latter, the profiles for the different G.P.A. levels in 
relation to the different contrasts examined, do not differ 
substantially from those obtained using I.O. levels except for 
the fact that G.P.A. levels for the low and medium crroups are 
more sharply differentiated than for the low and medium I.Q. 
levels. It appears that in grading children, teachers are 
more prone to differentiate poor from medium achievers’, while 
I.Q. ^valuations tend to differentiate better among subjects 
at higher ability levels. This is a finding that might have 
some interest with reference to grading systems. It should 
be remembered that when analyzing the distribution, of G,P.A. 
ave.'ages it was found that the criterion for grading tended 
to be more strict for younger than for the older children. 

The performance of subjects at specified ages was studied 
in terms of both I.Q. levels and G.P.A. scores. Comparisons 
were made between the findings obtained in both cases when dis- 
cussing G.P.A. Icivels (Tables 23 to 34). The similarity of the 
results was noticed and specific d.'fferences were listed in the 
corresponding text. In general, significant contrasts became 
sharp or sharper with increasing age samples in VA and VB prob- 
lems, while P problems seem to lose differentiating power at 
around 10 years of age. It is around 9 years of age that a 
transition occurs for the picture problems. In subsequent 
years when they are 10 and 11 years of age, a similar transi- 
tion occurs for the verbal problems. 
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In summary it can be said that there is improvement of 
performance in the successive years, that subjects are able 
to differentiate between logical structures , and that the 
language used influences the performance in the problems. 
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APPENDIX h 



Jimmy ‘s grandfather gave Jimmy some big and small balls. 
They were of two colors, blue and gray. Jimmy misplaced one 
kind. Which one did he misplace — the big or the small, e 

blue or the gray? 



Questions* 


Answers 


1 . Airplane 


No 


2. Big gray balls 


No 


3. Black balls 


No 


4. Big and small gray balls. 


No 


5 . Wagons 


No 


6 . Boat 


No 


7 . Yellow balls 


No 


8. Small gray balls 


No 


9 . Ponies 


No 


10. Big blue -balls 


No 



♦In all the P problems the "questions" are drawings pre- 
senced on cards. The description of the corresponding 
drawing is given next to the number of the card. 
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Bob went to a boat show. There were 3 kinds of boats, 
passenger boats, sail boats and row boats. They were either 
gray or blue. One of the beats got a prize. Which one was 
that — the passenger, sail or row, gray or blue? 





Questions 


Answers 


1. 


One blue and one gray 


No 


2. 


Yellow sail boats 


No 


3. 


Blue sail boats 


No 


4. 


A gray pass anger boat 


No 


5. 


A blue row boat 


No 


6. 


A gray sail boat 


\ 

No 


7. 


Airplanes 


No 


8. 


A blue passenger boat 


No 


9. 


Blue cars 


No 


10. 


1 

A blue and a gray sail boat. 


No 
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Mark had saved enough money to buy a new toy, so he 
went to the toy store. He saw boats, cars and trucks. 



They were either white or gray or black, 
buy only one. ''?hich one did he buy? 


He wanted to- 




Questions 


Aiswers 


1. 


White, gray, black boats 


No 


2. 


A chair 


No 


3. 


A white car 


No 


4. 


White, gray, black cars 


No 


5. 


Horses 


No 


6. 


Wagons 


No 


7. 


White and black trucks 


No 


8. 


A yellow boat 


No 


9. 


A white boat and white car 


No 


10. 


A gray boat 


No 
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There are two kinds of trains, passanger and freight. 
They have either two cars or six cars. I am thinking of 
one of these. Could you tell me which one I am thinking 

about? 



Questions 


Answers 


1. A passanger and a freight train 
with 4 cars each 


No 


2. A passanger-freight train with 


No 


5 cars. 




3. A 2-car and a- 6-car freight 


No 


train 




4 . An airplane 


No 


5. A freight train with 2 cars 


No 


6. A freight train with 6 cars 


No 


7. A passanger train with 2 car.s 


No 


8. An automobile 


No 


9. A passanger and a freight train 
both with 8 cars 


No 


10 . A bus 


No 
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John had two kinds of squirt guns — big and little. 
They were yellow squirt guns and black squirt guns. He 
gave one of these to his little brother. Which one did 
he give — the black or the yellow# the big or the little? 



Questions Answers 

1. A little blue squirt gun No 

2. A little yellow squirt gun - No 

3. A large and a small yellow squirt gun No -c 

4. A large and a small orange squirt gun No 

5. A little black squirt gun No 

6. A bow and an arrow No 

7. A black and a yellow large squirt gun No 

8. A big black squirt gun No 

9. A big and a small black squirt gun No 

10. A big yellow squirt gun No 
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Bill wanted something cool for the afternoon snack. So 
he v/ent to the ice cream shop, ht the shop they had ice cream 
Tones, snow cones and popsickles. They had 3 flavors, straw- 
berry, lemon and chocolate. Which one did Bill buy - an ice 
<;now cone or a popsickle, and what flavor. 



nream cone 



Questions 

1. Ice cream cone 

2 . An orange popsickle 

3 . A lemon ice cream cone, a straw- 
Ju 0 j-j-y popsickle and a blueberry 
snow cone 

4. A chocolate ice cream cone 

5. Popsickles: orange, lemon 

strawberry emd blueberry 

6. A strvav.’berry snow cone 

7. A strawberry and a lemon snow cone 

8 . A blueberry popsickle 

9. A blueberry snow cone 

10. Lemon, strawberry, blueberry and 
orange ice cream cones 



Answers 

No 

No 

No 

No 

No 

No 

No 

No 

No 

No 
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Don has 2 sets of blocks, big and small. There are some 
yellow and some blue. Yesterday he was playing with only one 
kind. Could you find out what kind — big or small, blue or 
yellow? 



Questions 
1 - An airplane 

2. Big yellow blocks 

3. Small orange blocks 

4. Large and small yellow 

5 . Wagons 

6. h gray boat 

7. Little blue blocks 

8. Little yellow blocks 

9. Horses 

10. Big brown blocks 



Answers 

No 

No 

No 

blocks No 

No 

No 

No 

No 

No 

No 
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John had 3 kinds of boats, canoes, sail boats and row 
boats. They were brown and black. He lost one of these 
boats. Which one did he lose — a brown boat or a black 
boat, a canoe or a row boat, or a sail boat? 





Questions 


Answers 


1. 


A brown and a black canoe 


No 


2. 


Yellow sail boats 


No 


3. 


Brown sail boats 


No 


4. 


h black canoe 


No 


5. 


A brown row boat 


No 


6. 


A black sail boat 


No 


7. 


Airplanes 


No 


8. 


A brown canoe 


No 


9. 


Blue cars 


No 


10. 


A brown and a black sail boat 


No 
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Tom has 3 kinds of automobiles — buses, trucks and 
cars. They are white, gray and black. He uses. one of 
these for his personal use. Which one does he use — 
the car, bus or the truck and of what color, gray, black 
or white? 



Questions 

1. Black and gray buses 

2 . A chair 

3. A white car 

4. White, gray and black cars 

5. A horse 

6 . VJagons 

7. White, gray and black truck 

8. A yellov; bus 

9. A gray bus and a white car 

10. A gray truck 



Answers 

No 

No 

No 

No 

No 

No 

No 

No 

No 

No 
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3lVAi 

Instructions and Corresponding Questions and Answers 

John has 20 horses. There are black race horses and white race 
horses. There are black iarm horses and white farm horses. I wsjit 
you to figure out how many black farm horses there are? 

Questions Answers 



1. 


How many horses- does John ride? 


1. 


10. 


2. 


How many white horses does John have? 


2. 


7. 


3. 


How many brown horses does John 
have? • 


3. 


0. 


k. 


How many white racing horses does 
John have? 


k. 


5. 


5. 


How many black racing horses does 
John have? 


5. 


5. 


6. 


How many brown racing horses does 
John have? 


6. 


0. 


T. 


How many white farm horses does 
John have? 


7. 


2. 


8. 


How many brown farm horses does 
John have? 


8. 


0. 


9. 


How many horses did John sell? 


9. 


0. 


10. 


How many ponies does John have? 


10. 


0. 
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Instructions and Corresponding Questions and Answers 

Daddy bought Jirauiy ^0 trucks. Some were red, some blue and some 
green. Some were fire trucks and some were array trucks. How many 
green army trucks did Daddy buy Jimmy. 







Questions 


Answers 


i. 


How many 
himself? 


cars does Daddy have for 


1. 1. 


2. 


How 


many 


cars did Daddy buy Jimmy? 


2. 0. 


3. 


How 


many 


red trucks were there? 


3. 15. 


k. 


How 


many 


blue trucks were there? 


4. IT. 


5. 


How 


many 


red fire trucks were there? 


5. 3. 


6. 


How 


many 


red army trucks were there? 


b. 12. 


T. 


How 


many 


blue fire trucks were there? 


7. 7. 


8. 


How 


many 


blue army trucks were there? 


8. 10. 


9. 


How 


many 


green fire trucks •;’'ere there? 


9. 7. 


10. 


How many 
Jimmy? 


brown trucks did Daddy buy 


10. 0. 
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Instructions and Corresponding Questions and Ansi^ers 



Joe and his two firiends Peter and Mark went to the store to 
"buy some marbles. Each one of them bought some green ones, some 
red ones, and some blue ones. Altogether they bought ^5 marbles. 



How 


many blue marbles did Mark biQr? 








Questions 




Answers 


1. 


How many green marbles did the three 
of them buy? 


1. 


15. 


2. 


How many red marbles and green marbles 
did Peter buy? 


2. 


10. 


3. 


Did they use the marbles right away? 


3. 


Yes. 


k. 


How many green marbles did Mark buy? 


k. 


5. 


5. 


How many red marbles did Peter buy? 


5. 


5. 


6. 


Did Peter buy more marbles than Joe? 


6. 


No. 


7. 


Are the red marbles larger than the 
green ones? 


7. 


No. 


8. 


How many blue marbles did Jow and 
Peter buy? 


8. 


10. 


9. 


Did they buy anything else besides 
marbles? 


9. 


No. 


10. 


How many red marbles did the three 
of them buy? 


10. 


15. 
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Instructions and Corresponding Questions and Answers 

Last Sunday the second graders had a party for all the hoys and 
girls in the class. There were 10 hoys and girls at the party and 
some wanted orange pop and some root heer. How many hoys drank root 
heer? 





Questions 


Answers 


1. 


Did all the hoys and girls go to 
the party? 


1. 


Yes. 


2. 


How many girls dranlc root heer? 


2. 


2. 


3. 


How many bottles of pop did they 
have at the party? 


3. 


2k. 


k. 


Was the '-l acher at the party? 


k. 


Yes . 


5. 


Was the p rty in the morning or 
in the afternoon? 


5. 


Afternoon 


6. 


How many girls were at the party? 


6. 


6. 


7. 


Did they play games at the party? 


7. 


Yes. 


8. 


How many girls drank orange pop? 


8. 


k. 


9. 


Were there more hoys than girls 
at the party? 


9. 


No. 


10. 


How many hoys drank orange pop? 


10. 


1. 
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Instructions and Corresponding Questions and Answers 

David and his two friends Boh and Dan decided to go ■Cishing. 
Each of the hoys caught sene blue gils and some perch. At the 
end of the day the hoys had caught a total of 15 fish. How many 
hlue gils did David catch? 

Questions Answers 



1. 


How many perch did Boh catch? 


1. 


2. 


2. 


\Jho caught the most fish? 


2. 


Each hoy caught the 
same nuraher of fish 


3. 


How manv fish did Boh catch? 


3. 


5. 


4. 


How many hull heads did they catch? 


4. 


0. 


5. 


VJhat kind of fishing poles did they 
use? 


5. 


Cane poles. 


6. 


How many hlue gils did Dan catch? 


6. 


3. 


7. 


^'/hat kind of hait did they use? 


7. 


Minnows . 


8. 


How manv fish did Dan catch? 


8. 


5. 


9. 


How much time did the hoys spend 
fishing? 


9. 


4 hours. 


10. 


How many perch did David catch? 


10. 


2. 
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Instructions and Corresponding Questions and Answers 

Joe and Jack and Jim each play on the same basehall team* At the 
end of the season the three hoys found tnat together they had a total 
of 90 home runs, walks and strikeouts. How many home runs did Jack 
have? 





Questions 




Answers 


1. 


How many walks did Jim have? 


1. 


10. 


2. 


How many home runs , walks and 
strikeouts did Jim have? 


2. 


30. 


3. 


How many strikeouts did Jack have? 


3. 


10. 


h. 


Did each have the same nuiiiher of 
walks? 


h. 


No. 


5. 


How many walks and strikeouts did 
Jack have? 


5. 


20. 


6. 


Who had the most strikeouts? 


6. 


Joe. 


T. 


How many home runs did Joe have? 


7. 


10. 


8. 


Who had the most walks? 


8. 


Joe and Jack each 
had 10. 


9. 


How many strikeouts , walks and 
home runs did Joe have. 


9. 


30. 


10. 


Who had the most homeruiis? 


10. 


Jim. 
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Instructions and Corresponding Questions and Answers 

A.t the dog show this year, there were 20 dogs entered. There 
were some large hlack dogs and some large brown dogs; there were 
also some small black dogs and small brown dogs. See if you can 
figure out how many large black dogs there were in the show. 



Questions 



Answers 



1. 


What day was the dog show? 


1. 


Satur 


2. 


How many small dogs were there 
in the show? 


2. 


7. 


3. 


How many white dogs were there? 


3. 


None. 


k. 


How many small black dogs were 
there? 


h. 


5. 


5. 


How many large brown dogs were 
there? 


5. 


5. 


6. 


Were there any poodles? 


6. 


Yes. 


7. 


How many small brown dogs were 
there? 


7. 


2. 


8. 


How many small white dogs were 
in the show? 


8. 


None . 


9. 


How man^/’ prises were given? 


9. 


3. 


10. 


How many large white dogs were 
there? 


10. 


None. 
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Instructions and Corresponding Questions and Answers 

John and Billy went to the carnival and bought tickets for 
fides. Some were for the Ferris Wheel, some for the Roller Coaster, 
and some for the Merry-go-Round. They each bought some tickets for 
all three rides. How many tickets did John get for the Merry-go- 
Round? 



Questions Answers 



1. 


Did John's dad go with them to 
the carni'val? 


1. 


Yes. 


2. 


How much popcorn did they buy at 
the carnival? 


2. 


They each had one box. 


3. 


How many Fei-ris \^eel tickets did 
the two of them buy? 


3. 


ik. 


k. 


How many Roller Coaster tickets did 
the boys buy? 


h. 


l6. 


5. 


How many Ferris \^heel tickets did 
Billy buy? 


5. 


8. 


6. 


How many Ferris Wheel tickets did 
John buy? 


6. 


6 . 


T. 


How many Roller Coaster tickets did 
Billy buy? 


T. 


6. 


8 . 


How many Roller Coaster tickets did 
John buy? 


8. 


10. 


9. 


How many Merry-go-Round tickets did 
Billy buy? 


9. 


k. 


10. 


How many Bingo games did John's dad 
play? 


10. 


3. 
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Instrtictions and Corresponding Questions and Answers 

Jiiaisy and his friends decided to count the commercials they saw 
on TV for one week. The shows they watched were "Flipper", "Lassie", 
and "Batman". Each siiow had some commercials for toothpaste, some 
for cereal, and some for candy. Altogether they counted ^5 commercials. 
How many candy commercials did they see on "Batman". ' ' 



1 . 

2 . 

3. 

k. 

5. 

6 . 

7. 

8 . 

9. 



Questions 




Answers 


How many cereal commercials were there 
altogether? 


1. 


20. 


How many cereal and toothpaste commercials 
did they see o:i “Flipper"? 


2. 


10. 


What day did they start counting the 
commercials? 


3. 


Sunday. 


How many cereal commercials were on 
"Batman"? 


4. 


10. 


Plow many toctlipaste commercials were 
on "Flipper'*? 


5. 


2. 


Were there more commercials on "Batman" 
than on "Lassie"? 


6. 


Yes. 


Were the toothpaste commercials Letter 
than the cereal commercials? 


T. 


No. 


How meny candy commercials were on 
"Flipper" and "Lassie" together? 


8. 


11. 


Which was their favorite show? 


9. 


"Lassie 



10. How many toothpaste commericals were 10. 10. 

there in all? 
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Instructions and Corrosponding Quastions and Ansvars 

Wa have 50 objects v'alled C. There are two kinds of C’s, one 
kind is called B, the ether kind is called G. Any B can be either 
a R or a T, and any C- can be either a R or a T. No B can be a G and 
no R can be a T. No B can be a G and no R can be a T. Will you find 
out how many of the G objects are also called T? 





Questions 




Answers 


1. 


How many K’s are there? 


1. 


11. 


2. 


How many R objects are also 
called G? 


2. 


15. 


3. 


How many T objects are also 
called B? 


3. 


10. 


k. 


How many N objects are there? 


k. 


10. 


5. 


How much is K times C? 


5. 


550. 


6. 


Are there more G than B objects? 


6. 


No. 


7. 


How many R objects are there? 


T. 


35. 


8. 


Are there more R objects than 
T objects? 


8. 


Yes. 


9. 


Are there any objects called M? 


9. 


No. 


10. 


How many R objects are also 
called B? 


10. 


20. 



123 - 



33VB^ OSVB^) 

Instructions and Corresponding Questions and Answers 

There are UO objects called T. There are two kinds of T objects > 
one kind is called E and the other kind is called F. Each F object is 
either an A, a S or a P. Also each E object is either an A, a S or a 
P. No E object is called F and no F objects are called E. Also no A 
object is called S or P, no S object is called A or P and no P object 



is 


called S or A. IIow loany A objects are 


also 


called E. 




Quustions 




Answers 


1. 


Are any objects called K? 


1. 


No. 


2. 


How mar^ E objects are also called 
P? 


2. 


15. 


3. 


How many objects are called S? 


3. 


10. 


k. 


How many E objects are also 
called S? 


4. 


4. 


5. 


How many objects are called P? 


5. 


20. 


6. 


How many F objects are also called 
A? 


6. 


8. 


7. 


How many F objects are also called 
P? 


7. 


5. 


8. 


How many E objects are also called 
W? 


8. 


0. 


9. 


How many F objects are also called 
S? 


9. 


6. 


10. 


Are there more P objects than S 
objects? 


10. 


Yes. 
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Instructions and Corresponding Questions and Answers 



We have three kinds of T objects. One kind is called M, 
another kind is called N, and another kind is called P. Further, 
each M, N, or P can also be called either a Q, a R, or an S. Al- 
together there are fifty objects. How many of the N objects are 
also called S? 

Questions Answers 



1 . 


How 

are 


many Q objects and R objects 
called P? 


1 . 


15 . 


2 . 


How many M objects and P objects tre 
also called S? 


2 . 


5 . 


3 . 


Are 


there more Q objects than S objects? 


3 . 


Yes . 


k. 


How 


many H objects are called Q? 


k. 


5 . 


5 . 


How 


many objects are called Q? 


5 . 


25 . 


6. 


How 


many M objects are called A? 


6. 


0 . 


7 . 


How 


many objects are called R? 


7 . 


15 . 


8. 


Are 


there more P objects than R objects? . 


8. 


Yes. 


9 . 


How 


many objects are called K? 


9 . 


0 . 


10 . 


How 


many P objects are also called R? 


10 . 


5 . 
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Instructions ar^d Corresponding Questions and Answers 

We lave objects called S. There are two kinds of S*s. One 
kind is called P; the other kind is called Q. Any P can be an A or 
a B, and any Q can be an A or a No P can be a Q, and no A can be 
a B. Will you find out how many of the Q objects are also called B? 

Questions Answers 



1. 


How many K's are there? 


1. 


7. 


2. 


How many A objects are also called 
Q? 


2. 


8. 


3 . 


How many B objects are also called 
P? 


3 . 


l4. 


4. 


How many N objects are there? 


4. 


5 . 


5 . 


How much is K times S? 


5 . 


280 . 


6. 


Are there more Q than P objects? 


6. 


No. 


r. 


How many A objects are there? 


T. 


l4. 


8. 


Are there more A objects than B 
objects? 


8. 


No. 


9. 


Are there any objects called M? 


9. 


No. 


10. 


How many A objects are £ilso 
called P? 


10. 


6. 
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Instructions and Corresponding Questions and Answers 

There are 30 objects called N, There are two kinds of objects, 
one* kind is called H and thr other kind is called L. Each L object is 
either an X, a Y or a Z, .‘..so each H object is either an X, a Y or a 
Z. Wo H object is called and no L objects are called H. Also no X 
object is called Y or Z, no Y object is called X or Z, and no Z object 
is called Y or X. How many X objects are also called H? 



Questions 

1. ^re any objects called K? 

2. How many H objects are also called Z? 

3. How many objects are called Y? 

U. How niP.ny.H objects are also called Y? 

5. How many objects are called Z? 

6. How man/-L objects are also called X? 

7. How nuny 'L objects are also ca3.1ed Z? 

8. How many K objects are also called W? 

9. How many L objects are also called Y? 

10. Are there more Z objects than Y 
objects? 



Answers 



1. No. 

2 . 3 . 

3. 13. 
h. T. 

5. 5. 

6 . 8 . 

T. 2. 
8 . 0 . 
9 . 6 . 

10. No. 
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Instructions and Corresponding Questions and .%iswers 

We have three kinds of G objects. One kind is called A, another 
kind is called B, and another kind is called C. Further, each A, B, 
or C can also be called either an M, an N, or a Q. /iltogether there 
are sixty objects. How many of the B objects are also called Q? 





Questions 


Answers 


1* 


How many M objects and H objects are 
called C? 


1. 


Ik, 


2. 


How many A objects and C objects are 
also called Q? 


2. 


7. 


3. 


Are there more M objects than Q objects? 


3. 


Yes. 


k. 


How many B objects are called M? 


k. 


7. 


5. 


How many objects are called M? 


5. 


32. 


6. 


How many A objects are called F? 


6. 


0. 


7. 


How many objects are called K? 


7. 


Ik, 


8. 


Are there more C objects than W objects? 


8. 


Yes. 


9. 


How many objects are called H? 


9. 


0. 


10. 


How many C objects are also called N? 


10. 


2. 
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APPENDIX B 



The graphs presented here illustrate the multivariate 
results given in Tables 21 and 22. Symbols to the left of 
each curve identify a specific Age or I.Q. group; those to 
the right, the size of the Scimple. The parallelism statistic, 
9 indicates in all these cases that parallelism could not be 
rejected at the .05 level. Significance levels for F-ratios 
eind Scheff4 Comparisons are denoted for the .10 level of 
significance, "**'* for .05, and •'***” for .01. F is the 
F-ratio for ages or I.Q. levels, and Fg for the scales (prob- 
lems or administrations.) 
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